Closed Bug 248970 (PrivateBrowsing) Opened 20 years ago Closed 16 years ago

Private Browsing mode (global toggle for saving/caching everything)

Categories

(Firefox :: Private Browsing, enhancement)

enhancement
Not set
normal

Tracking

()

VERIFIED FIXED
Firefox 3.1b2

People

(Reporter: mozbug-mhade, Assigned: ehsan.akhgari)

References

(Depends on 1 open bug, Blocks 2 open bugs, )

Details

(4 keywords, Whiteboard: PRD:SPI-002a)

Attachments

(15 files, 66 obsolete files)

35.11 KB, patch
dietrich
: review+
Details | Diff | Splinter Review
24.92 KB, patch
mconnor
: review+
Details | Diff | Splinter Review
5.97 KB, patch
bzbarsky
: review+
bzbarsky
: superreview+
Details | Diff | Splinter Review
6.89 KB, patch
mconnor
: review+
Details | Diff | Splinter Review
7.32 KB, patch
mconnor
: review+
Details | Diff | Splinter Review
127.33 KB, patch
Details | Diff | Splinter Review
29.05 KB, patch
Dolske
: review+
Details | Diff | Splinter Review
1.04 KB, patch
dwitte
: review+
bzbarsky
: superreview+
Details | Diff | Splinter Review
12.27 KB, patch
dcamp
: review+
Details | Diff | Splinter Review
24.09 KB, patch
dcamp
: review+
Biesinger
: review+
Biesinger
: superreview+
Details | Diff | Splinter Review
47.20 KB, patch
mconnor
: review+
Details | Diff | Splinter Review
24.07 KB, patch
mconnor
: review+
mconnor
: superreview+
Details | Diff | Splinter Review
9.11 KB, patch
dietrich
: review+
Details | Diff | Splinter Review
23.58 KB, patch
zeniko
: review+
Details | Diff | Splinter Review
36.82 KB, patch
sdwilsh
: review+
Details | Diff | Splinter Review
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040615 Firefox/0.9
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040615 Firefox/0.9

In the next version of OS X, Safari will have a "Private Browsing" mode which,
basically, disables writing to cache, URL history, etc.
I think this is a good idea, and it's small and useful enough that it should go
into the browser, not an extension.

Reproducible: Always
Steps to Reproduce:
So basically, implement some sort of universal pref to toggle all of the various
cache/cookies/etc off without opening prefs?  I don't know if this deserves a
toggle in this manner, or how practical that is.

morphing summary to describe what this actually does in Safari (whenever Tiger
comes out)

The flipside to this is the plan to have Ctrl-Shift-C act as a global privacy
clear option, so that if you want to clear a certain group of things, you can
set it to do that.

Summary: Wish: Implement a "Private Browsing" mode → Implement a global toggle for privacy-related prefs
(In reply to comment #1)
No, more like a "don't remember what I'm doing now" toggle, which would allow
browsing sites without cookies/history/cache being written to. This would allow
users to do any privacy-sensitive browsing (online banking on a public computer,
shopping for gifts on the family computer, etc.) without having to completely
clear history/cache/cookies afterwards. This is more useful because when
clearing all privacy sensitive material you would loose for example login
cookies for other sites, the complete history which may contain useful items,
and all of the cache making browsing slower until the cache is refilled.
Privacy mode is also more complicated than a global privacy clear option, as it
means cookies and history would have to be moved to a two-layer memory/disk
model like cache, and individual items would need to have a flag that prevents
them to move from memory to disk. This is necessary beause the browser would
still need to handle cookies and history, just not write them to disk
completely. I'm not sure wether this can be done in an extension at all.
See also bug 155300 (guest profiles) and bug 103765 (anonymous mode).
Summary: Implement a global toggle for privacy-related prefs → Private Browsing mode (global toggle for saving/caching everything)
*** Bug 260266 has been marked as a duplicate of this bug. ***
would be a great addition for Firefox, esp. if you have to use a public Terminal

(and it would be great for the Mac 1.0 to have this feature before safari gets it)

btw: shouldnt this be moved to browser, since the suite could use this too?
If Browser needs a bug for it too, great, file a bug there.  No one still
working/triaging Seamonkey really cares about Firefox.
Status: UNCONFIRMED → NEW
Ever confirmed: true
It is an extremely useful feature specifically for internet cafes or machines
which are used by multiple users with the same login or even for the security
paranoid. It is not that difficult to implement either. Definately something I
would be keen to see and would be one less feature safari has that we don't.
Assignee: firefox → nobody
QA Contact: bugzilla → menus
FWIW, this feature has been seperately implemented by both the Stealther and Distrust extensions.

Which means someone can probably close this bug.
(In reply to comment #9)
> Which means someone can probably close this bug.
second.
(In reply to comment #9)
> FWIW, this feature has been seperately implemented by both the Stealther and
> Distrust extensions.
> 
> Which means someone can probably close this bug.

Since when should bugs be closed because there is an extension for it?

I'll take this. I managed to get a working mode today (purely back-end and pref based, no UI just yet), and I'll post the patch in the morning once I find suitable reviewers.
Status: NEW → ASSIGNED
Whiteboard: PRD:SPI-002a
Assignee: nobody → ventnor.bugzilla
Status: ASSIGNED → NEW
Attached patch First try (obsolete) — Splinter Review
If Private Browsing ("privacy.privatebrowsing") is turned on:

* Cache won't be touched.
* Nothing will be added to Places history.
* No new entries will be saved in autocomplete.
* Download records will delete themselves when the download is complete.
* Cookies are kept until Private Browsing is turned off, or the browser closes, at which point the cookies will be deleted.

I can almost guarantee this won't break anything, because all this does is it sneakily manipulates pref-check functions and return values in each section to do the right thing.
Attachment #267216 - Flags: superreview?(bzbarsky)
Attachment #267216 - Flags: review?(vladimir)
> I can almost guarantee this won't break anything

view-source and meta charset reloads come to mind as things it could break.  Also wyciwyg:.  I assume you've tested all of these?  I assume you've also tested the performance impact of getting the pref every single time and determined that it's cheap enough that caching it is not worth it?

This probably also breaks visited link coloring, unless I'm missing something.

In any case, I'm not going to be able to review this thoroughly in a reasonable timeframe.  Please ask someone else?  biesi might be a good idea.
Comment on attachment 267216 [details] [diff] [review]
First try

(In reply to comment #14)
> view-source and meta charset reloads come to mind as things it could break.

view-source works.

> Also wyciwyg:.  I assume you've tested all of these? 

I haven't yet tested meta charset reloads or wyciwyg, I'm not really sure how to test them. The reason for my (over)confidence is that this patch only really changes existing preference or policy checks.

> I assume you've also
> tested the performance impact of getting the pref every single time and
> determined that it's cheap enough that caching it is not worth it?

I can't find an impact at all. Some of the functions which I change already seem to check a pref every time they're called.

> This probably also breaks visited link coloring, unless I'm missing something.

No, this does as expected (expected IMO being: visited link colours show up when a link is visited outside of Private Browsing, but not when its visited while turned on).

> In any case, I'm not going to be able to review this thoroughly in a reasonable
> timeframe.  Please ask someone else?  biesi might be a good idea.
> 

OK.
Attachment #267216 - Flags: superreview?(bzbarsky) → superreview?(cbiesinger)
(In reply to comment #15)
> I can't find an impact at all. Some of the functions which I change already
> seem to check a pref every time they're called.

I actually forgot to mention the primary reason, many existing pref listeners in each section of Mozilla that this patch touches have a branch (eg "browser.foo") and changing the existing nsIPrefBranch2, or implementing a separate one, for each section would be messier than just reading the pref when needed.
Comment on attachment 267216 [details] [diff] [review]
First try

I'm not the right person to review this, and I don't think biesi is the right sr -- you really want mconnor, at the very least.
Attachment #267216 - Flags: review?(vladimir)
Comment on attachment 267216 [details] [diff] [review]
First try

(In reply to comment #17)
> (From update of attachment 267216 [details] [diff] [review])
> I'm not the right person to review this, and I don't think biesi is the right
> sr -- you really want mconnor, at the very least.
> 

Thats precisely the problem I had before I submitted a patch like this, which touched different sections of Mozilla. From the list of reviewers at http://www.mozilla.org/hacking/reviewers.html you were under toolkit and biesi was under netwerk. 

I'll get mconnor.
Attachment #267216 - Flags: review?(mconnor)
Comment on attachment 267216 [details] [diff] [review]
First try


> PRInt32 
> nsDownloadManager::GetRetentionBehavior()
> {
>   // We use 0 as the default, which is "remove when done"
>   nsresult rv;
>   nsCOMPtr<nsIPrefBranch> pref = do_GetService(NS_PREFSERVICE_CONTRACTID, &rv);
>   NS_ENSURE_SUCCESS(rv, 0);
>+
>+  // The user is in private browsing mode, delete the download record when
>+  // completed
>+  PRBool privMode = PR_FALSE;
>+  pref->GetBoolPref("privacy.privatebrowsing", &privMode);
>+  if (privMode)
>+    return 0;
Please declare a constant at the top of the file, just as everything else that uses a pref does in the file.

Also, this will require some UI changes if the download manager is already open:
http://mxr.mozilla.org/seamonkey/source/toolkit/mozapps/downloads/content/downloads.js#152
Attachment #267216 - Attachment is obsolete: true
Attachment #267216 - Flags: superreview?(cbiesinger)
Attachment #267216 - Flags: review?(mconnor)
So, I thought I'd commented here, but I didn't.  I think this is a good start, but there's some things I think need addressing:

* Toggling the mode on/off

Using prefs and/or pref observers seems kinda wrong, why not just have observers/notifications?

* Cookies

This is a pretty painful impl, since this would mean that you'll lose cookies that you had before you started in private browsing mode and accessed while you're there.  Fortunately, the cookie service now has a creationTime that would be useful here.  I think only clearing new cookies is acceptable for this feature, since stuff you've already been to will likely be in your history as well...

* History

It makes more sense to kill everything from the visits table, and remove any URLs that have zero visits.  Achieves the same thing, but means link coloring etc still work during the session.  I'm not sure if we can suppress flushing to disk, but if not I'm not as concerned about "must never touch magnetic media" as some are.

* Cache

Totally disabling cache feels like a mistake.  Why not just disable the disk cache, and clear memory cache before re-enabling disk cache?  (biesi, does disabling the disk cache preclude reloading from disk on exit of private browsing?)

* Downlaods

I bet sdwilsh's comments are bang on, so I'll stay quiet.
Status: NEW → ASSIGNED
A non-programmer suggestion:

What would happen if a 'guest' temporary user were whipped up and the browser were restarted using that profile?  Would this sort of feature be more thorough or be a more "clean" way to implement a feature like this?

A restart isn't not as convenient as having a button/hotkey to enable/disable private browsing, but it might be something to think about.
(In reply to comment #22)
> A non-programmer suggestion:
> 
> What would happen if a 'guest' temporary user were whipped up and the browser
> were restarted using that profile?  Would this sort of feature be more thorough
> or be a more "clean" way to implement a feature like this?

That would not be this feature (see comment 0), since you wouldn't be able to access to any of your current bookmarks or history.

Note that this feature already exists in Safari, and it looks like we're going to try to implement it more or less the same way.
(In reply to comment #21)
> * Toggling the mode on/off
> 
> Using prefs and/or pref observers seems kinda wrong, why not just have
> observers/notifications?

It'd be more code, but I suppose that's best.

> * Cookies
> 
> This is a pretty painful impl, since this would mean that you'll lose cookies
> that you had before you started in private browsing mode and accessed while
> you're there.  Fortunately, the cookie service now has a creationTime that
> would be useful here.  I think only clearing new cookies is acceptable for this
> feature, since stuff you've already been to will likely be in your history as
> well...

I wrote this patch before the cookie rewrite ;) I wanted to do that but couldn't before because creation "time" was just an incremented counter. I can do that now that the DB uses a proper creationTime.

> * History
> 
> It makes more sense to kill everything from the visits table, and remove any
> URLs that have zero visits.  Achieves the same thing, but means link coloring
> etc still work during the session.  I'm not sure if we can suppress flushing to
> disk, but if not I'm not as concerned about "must never touch magnetic media"
> as some are.

Put on a tinfoil hat for a sec. Do we want link colouring to work? It can be used as another method to trace your steps...

> * Cache
> 
> Totally disabling cache feels like a mistake.  Why not just disable the disk
> cache, and clear memory cache before re-enabling disk cache?  (biesi, does
> disabling the disk cache preclude reloading from disk on exit of private
> browsing?)

I don't know much about how cache works, that just seemed like an easy way to go.

> * Downlaods
> 
> I bet sdwilsh's comments are bang on, so I'll stay quiet.

Like I said, I should probably know more about the code I'm touching before I touch it ;)
That being said, this isn't a top priority for me right now as I have other bugs I really want fixed. I wouldn't object if anyone else wanted to take this in the meantime.

> Put on a tinfoil hat for a sec. Do we want link colouring to work? It can be
> used as another method to trace your steps...

I think the idea is to make link coloring work, but just during the private browsing session.  After the session, that (temporary) history data goes away, and the links are no longer marked as visited.
(In reply to comment #25)
> > Put on a tinfoil hat for a sec. Do we want link colouring to work? It can be
> > used as another method to trace your steps...
> 
> I think the idea is to make link coloring work, but just during the private
> browsing session.  After the session, that (temporary) history data goes away,
> and the links are no longer marked as visited.
> 

I know, but I was wondering if we even want that....
Michael, thanks for working on this bug, this is a feature I would personally very much like to see land in Firefox 3.

I have a few questions about the implementation that might impact the UI:

1) Are we still considering launching a new instance, which would potentially enable us to display Firefox with a modified theme?
http://wiki.mozilla.org/PrivateBrowsing

2) If not, would it be possible to modify the color and opacity of the current theme, similar to how Personas for Firefox doesn't require a restart?
http://www.puffinlabs.com/personas/

There hasn't been much activity on this bug over the last month, can anyone give me an update on what the status is?
(In reply to comment #27)
> 1) Are we still considering launching a new instance, which would potentially
> enable us to display Firefox with a modified theme?
> http://wiki.mozilla.org/PrivateBrowsing
> 
> 2) If not, would it be possible to modify the color and opacity of the current
> theme, similar to how Personas for Firefox doesn't require a restart?
> http://www.puffinlabs.com/personas/

Alex, Madhava, Connor and I discussed this in various fora, and I think we've actually agreed that heavy modifications to the theme on entry of Private Browsing mode would probably be antithetical to its purpose, as it would call attention to the very fact that the user had entered a mode all about being unobserved.
Assignee: ventnor.bugzilla → nobody
Status: ASSIGNED → NEW
I'm going to give this one a serious shot...  :-)
Assignee: nobody → ehsan.akhgari
Target Milestone: --- → Firefox 3 M11
Version: unspecified → Trunk
Status: NEW → ASSIGNED
(In reply to comment #19)
> Also, this will require some UI changes if the download manager is already
> open:
> http://mxr.mozilla.org/seamonkey/source/toolkit/mozapps/downloads/content/downloads.js#152
> 

Shawn: I'm afraid that the link above may not point to what you intended to say any more.  Would you mind re-stating your concern in comment 19 so that I can make sure to consider it?

Thanks!
(In reply to comment #30)
> Shawn: I'm afraid that the link above may not point to what you intended to say
> any more.

fyi, you can use bonsai to figure it out - load up the cvs log for the file:
http://bonsai.mozilla.org/cvslog.cgi?file=/mozilla/toolkit/mozapps/downloads/content/downloads.js&rev=HEAD&mark=1.130
find the appropriate revision based on date, click on the linkified revision number, click on 'blame' at the bottom of the diff screen, then find the appropriate line number.

for this reason, it's always better to provide bonsai links to a particular revision & line number - that way, they don't change with time. ;)
(In reply to comment #21)
> So, I thought I'd commented here, but I didn't.  I think this is a good start,
> but there's some things I think need addressing:
> 
> * Toggling the mode on/off
> 
> Using prefs and/or pref observers seems kinda wrong, why not just have
> observers/notifications?

Hmm, using prefs, users can switch to private browsing mode, close the browser,
and open it and continue in the private browsing mode...  Unless of course we
don't want that; but not saving this value for next startups seems inconsistent
with what we do in other parts of the UI.

Example: we put a check mark besides View > Status Bar to indicate that the
status bar is shown.  We save this value for next startups.  If we put a check
mark next to the "Private Browsing Mode" menu item, the users would expect the
"check mark" to be preserved in the next startup, and may assume that it is and
gets bitten later on when she discovers that this value has not been saved.

Or am I totally missing something here?

> * Cookies
> 
> This is a pretty painful impl, since this would mean that you'll lose cookies
> that you had before you started in private browsing mode and accessed while
> you're there.  Fortunately, the cookie service now has a creationTime that
> would be useful here.  I think only clearing new cookies is acceptable for this
> feature, since stuff you've already been to will likely be in your history as
> well...

I should delve into some code to get how I should use the creationTime value...

> * History
> 
> It makes more sense to kill everything from the visits table, and remove any
> URLs that have zero visits.  Achieves the same thing, but means link coloring
> etc still work during the session.  I'm not sure if we can suppress flushing to
> disk, but if not I'm not as concerned about "must never touch magnetic media"
> as some are.

Two concerns:

1. The goal here is to respect user's privacy.  Suppose that the user enters
the private browsing mode, and performs a Google search and navigates to one of
the results in a new tab.  Now, if someone sees the Google results page over
the user's shoulders, they would immediately figure out what page the user has
visited.  If I want to get over-paranoid, I could even imagine them to memorize
the page title and then do a Google search of their own to find out exactly
what the user has viewed in their private browsing mode...

2. A more serious issue: if we write the URLs to the Places DB, with the
intention of clearing them off later, and in between we crash or get closed, on
the next startup, the URLs visited in the *private* mode exist in the DB (and
we guarantee that the DB survives our crash...).  Even if we add code to handle
that in the startup, what happens if someone takes the raw sqlite DB file from
the user's profile, and open it in their own app/viewer?

The paragraph I quote below from <http://wiki.mozilla.org/PrivateBrowsing>
seems to be very relevant here:

"By choosing to write *some* data to disk (perhaps in an encrypted format) we
have broken a clear and easy to understand contract between Firefox and the
user. The user / security expert will not be sure that there is no security
risk."

We don't want that, I guess.  Not getting visited link colors is something that
the ordinary user may not even notice (there are a lot of sites which disable
our default coloring via CSS) and the advanced user will appreciate (see point
1 above).  And we can document that this is intentional in the user docs for
the private browsing feature...

> * Cache
> 
> Totally disabling cache feels like a mistake.  Why not just disable the disk
> cache, and clear memory cache before re-enabling disk cache?  (biesi, does
> disabling the disk cache preclude reloading from disk on exit of private
> browsing?)

I guess this is the right approach here...
(In reply to comment #31)
> (In reply to comment #30)
> > Shawn: I'm afraid that the link above may not point to what you intended to say
> > any more.
> 
> fyi, you can use bonsai to figure it out - load up the cvs log for the file:
> http://bonsai.mozilla.org/cvslog.cgi?file=/mozilla/toolkit/mozapps/downloads/content/downloads.js&rev=HEAD&mark=1.130
> find the appropriate revision based on date, click on the linkified revision
> number, click on 'blame' at the bottom of the diff screen, then find the
> appropriate line number.

Thanks for the tip!

> for this reason, it's always better to provide bonsai links to a particular
> revision & line number - that way, they don't change with time. ;)

Agreed!  For future reference, here's the bonsai link to the code Shawn was talking about: <http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/toolkit/mozapps/downloads/content/downloads.js&rev=1.60&root=/cvsroot#152>
(In reply to comment #28)
> (In reply to comment #27)
> > 1) Are we still considering launching a new instance, which would potentially
> > enable us to display Firefox with a modified theme?
> > http://wiki.mozilla.org/PrivateBrowsing
> > 
> > 2) If not, would it be possible to modify the color and opacity of the current
> > theme, similar to how Personas for Firefox doesn't require a restart?
> > http://www.puffinlabs.com/personas/
> 
> Alex, Madhava, Connor and I discussed this in various fora, and I think we've
> actually agreed that heavy modifications to the theme on entry of Private
> Browsing mode would probably be antithetical to its purpose, as it would call
> attention to the very fact that the user had entered a mode all about being
> unobserved.

I agree.  In fact, I think that having a small icon in the status bar would be sufficient (if we even decide to have *that*), plus a notification (as mentioned in <http://wiki.mozilla.org/PrivateBrowsing#Making_Sure_the_User_has_the_Correct_Mental_Model> on the point when the user enters (or leaves) the private browsing mode.
Alias: PrivateBrowsing
Attached patch WIP Patch 1 - Cache Completed (obsolete) — Splinter Review
Firstly, thanks to Michael Ventnor for his work on this bug which allowed me to take over and build upon it.

Here is the first version of my WIP.  This is an unbitrotted version of Michael's patch.  I have completed the implementation regarding disk cache in this patch according to mconnor's comments.

This patch disables the disk and offline cache devices when the privacy.privatebrowsing pref is set to true, and re-enables them when it's set to false.  Upon the change of this pref's value to false, the code empties the memory cache, so that the items entered into the cache during user's private browsing period are no longer accessible after exiting this mode.  No data is written to disk when the user is in private browsing mode.
(In reply to comment #19)
> Also, this will require some UI changes if the download manager is already
> open:
> http://mxr.mozilla.org/seamonkey/source/toolkit/mozapps/downloads/content/downloads.js#152

Bug 394039 has reworked the download manager UI to get notified when a download is removed, so downloads.js no longer touches the browser.download.manager.retention pref, and the change you mentioned is no longer necessary.

(See the check-in for bug 394039: <http://bonsai.mozilla.org/cvsview2.cgi?diff_mode=context&whitespace_mode=show&file=downloads.js&branch=&root=/cvsroot&subdir=/mozilla/toolkit/mozapps/downloads/content&command=DIFF_FRAMESET&rev1=1.92&rev2=1.93>)
Since nsNavHistory::CanAddURI() is executed commonly, and since we expect privacy.privatebrowsing to change only occasionally, it makes sense to cache the value of this pref, thus speeding up nsNavHistory::CanAddURI().  This should address part of the concerns in comment 14.
Attachment #296499 - Attachment is obsolete: true
I'm really happy to see some activity in this bug.  Should we spin off a separate bug for designing the user interface for entering and exiting private browsing mode?
Yes.  I'm willing to work on that as well.  I'll file a bug and add the dependency here.

In the meamntime, I'd like to have your opinion on comment 34.  Thanks!
BTW, I'll do my best to land both the back-end and the UI for Beta 3, if I can get timely reviews.  I think it's important that this gets tested by users in Beta 3 and we get some user feedback about it.
The last time the ux team talked about private browsing we agreed that a small indicator to the left of the site button was the best way to go.  Let me check with Beltzner to see if I can give this some cycles, and I'll post some mockups. 
(In reply to comment #34)
> I agree.  In fact, I think that having a small icon in the status bar would be
> sufficient (if we even decide to have *that*), plus a notification (as
> mentioned in http://wiki.mozilla.org/PrivateBrowsing#Making_Sure_the_User_has_the_Correct_Mental_Model
>on the point when the user enters (or leaves) the private browsing mode.

Yup, I think that's right. The idea that Alex is referring to in comment #41 was that upon entering the mode a small indicator would appear somewhere near (or in?) the location bar that would also act as the "exit private browsing mode" button. Having it only in the status bar might not provide a good enough affordance for ending the "private mode" transaction.
Target Milestone: Firefox 3 M11 → Future
I filed bug 411929 to track the progress on the private browsing UI.  Some mockups would be great, Alex.  I agree that something near the location bar could be better noticed by the users.

BTW, Beltzner, did you intentionally change the Target Milestone field to Future?
This patch includes the same optimization (comment 37) to nsFormHistory-related part.
Attachment #296508 - Attachment is obsolete: true
(In reply to comment #24)
> > * Cookies
> > 
> > This is a pretty painful impl, since this would mean that you'll lose cookies
> > that you had before you started in private browsing mode and accessed while
> > you're there.  Fortunately, the cookie service now has a creationTime that
> > would be useful here.  I think only clearing new cookies is acceptable for this
> > feature, since stuff you've already been to will likely be in your history as
> > well...
> 
> I wrote this patch before the cookie rewrite ;) I wanted to do that but
> couldn't before because creation "time" was just an incremented counter. I can
> do that now that the DB uses a proper creationTime.

yep, this sounds good to me - note that we don't expose that information via the cookie interface yet, it'd live here:
http://mxr.mozilla.org/mozilla/source/netwerk/cookie/public/nsICookie2.idl

but file a bug against me if you want it, and i'll whip up a patch.
Depends on: 411952
(In reply to comment #45)
> yep, this sounds good to me - note that we don't expose that information via
> the cookie interface yet, it'd live here:
> http://mxr.mozilla.org/mozilla/source/netwerk/cookie/public/nsICookie2.idl
> 
> but file a bug against me if you want it, and i'll whip up a patch.

Done: bug 411952.  Thanks!

I'll modify the cookies related part of this patch as soon as you post a patch on that bug.  The updated IDL would suffice for me to get started.  :-)
SessionStoreService must also be modified to make sure the data for windows/tabs are not saved during the private browsing mode.  A patch containing this change will be posted shortly.
(In reply to comment #47)
Yeah, you'll probably want to make SessionStore behave as when .resume_from_crash is disabled (nothing will be written to disk) and at the end clean up similarly to what we do for "browser:purge-session-history"... and just ask me for review when you're ready.
Attached patch WIP Patch 4 - SessionStore (obsolete) — Splinter Review
(In reply to comment #48)
> (In reply to comment #47)
> Yeah, you'll probably want to make SessionStore behave as when
> .resume_from_crash is disabled (nothing will be written to disk) and at the end
> clean up similarly to what we do for "browser:purge-session-history"... and
> just ask me for review when you're ready.

This patch changes the SessionStore service to support privacy.privatebrowsing pref, as suggested above by Simon.

The user experience is summarized as below:

When the user enters private browsing mode, the entire session is considered private, so the SessionStore service gets is prevented from writing anything to the disk, and the existing sessionstore.js file is deleted from the user's profile.  The only services from the SessionStore available during this period are the window/tab undo features.  The SessionStore continues to collect information about open tabs/windows in the memory.

When the user leaves the private browsing mode, the entire data collected in the memory would be deleted (so that after leaving the private browsing mode, the closed windows/tabs from the sessions cannot be restored.  Then, the data for current windows/tabs are collected from scratch, and saved (if necessary) to disk.  This way, nothing done inside the private browsing mode can be reflected by the SessionStore service after leaving this mode.  The reason that the data for current windows/tabs are recollected is that the user should not consider any tabs/windows she leaves open before leaving the private browsing mode private.
Attachment #296562 - Attachment is obsolete: true
The Password Manager should not prompt the user to save new passwords or change already saved passwords in the private browsing mode.  I'll post a new WIP patch with this change shortly.
Should the "Private Browsing" mode use the normal cookies?

Reason for my question is: What exactly is this private browsing mode?

Just dont keep any local tracks during this session?
Or dont use my "Identity" (aka cookies) for the webpages during the session?
Or Both?
(In reply to comment #51)
> Should the "Private Browsing" mode use the normal cookies?

Yes.

> Reason for my question is: What exactly is this private browsing mode?

See <http://wiki.mozilla.org/PrivateBrowsing>.

Quote:
"The purpose of private browsing is to put Firefox into a temporary state where no information about the user's browsing session is stored locally."

Based on the above definition:

> Just dont keep any local tracks during this session?
> Or dont use my "Identity" (aka cookies) for the webpages during the session?
> Or Both?

The former is what Private Browsing is about.
This patch implements the changes suggested in comment 50.

The user experience is as follows:

While the user is in the private browsing mode, the password manager continues to fill in the usernames and passwords already stored in the user's profile, but does not prompt the user to save new authentication information in any manner.  After the user exits from the private browsing mode, the password manager will prompt her to store any new authentications as needed, just like before she entered the private browsing mode.
Attachment #296679 - Attachment is obsolete: true
>> Just dont keep any local tracks during this session?
>> Or dont use my "Identity" (aka cookies) for the webpages during the session?
>> Or Both?
>
>The former is what Private Browsing is about.

That's correct based on how we spec'd this out on the wiki page, but are we sure that this is the right behavior?  Here are two examples:

-Alex is shopping for an engagement ring, so he enters into private browsing mode and loads amazon.com.  Amazon recognizes Alex based on the cookies stored in his profile, and remembers that he spent the afternoon looking for engagement rings.  The next day Alex's soon to be fiancee sits down at his computer and loads up amazon.com (while not in private browsing mode) and sees a homepage full of "Other people who shopped for engagement rings looked at these rings as well! and Here is your recent shopping history!"

-Anna is trying to plan a surprise vacation for Alex.  Not knowing the implementation details of cookies, data stored on a server, and the way Firefox treats private browsing mode, she turns on private browsing mode and begins to research exotic tropical locations.  Using a cookie stored in her profile, Google Web History captures every search she does, not realizing that she is interested in privacy because she never logged out of her google account.  Alex sits down at Anna's computer and her Web History widget on her iGoogle home page exposes every search that she has done.

I can see users thinking that "privacy means privacy" and since they don't really get how all of this stuff works, situations like those two being rather common on shared computers.  This leads me to think that we should probably block access to cookies when the user is in private browsing mode.
What about the situation where a user wonders why they just got logged out of their Amazon account?
>What about the situation where a user wonders why they just got logged out of
>their Amazon account?

Yeah, this is why beltzner was in favor of maintaining access to cookies, because people won't get why nothing works when they are in private browsing mode.

Could we block access to exiting cookies, but allow the creation of new cookies in memory?  This would at least enable users to log back into amazon.com.  If they have to do the action of logging in, then maybe they won't be so upset when information gets tracked?

Neither solution is perfect, but I personally am leaning towards erring on the side of privacy.
(In reply to comment #54)
> I can see users thinking that "privacy means privacy" and since they don't
> really get how all of this stuff works, situations like those two being rather
> common on shared computers.  This leads me to think that we should probably
> block access to cookies when the user is in private browsing mode.

Hmm, interesting use cases.  Let's do an item by item analysis based on such a viewpoint.

We have the following items that should be affected by private browsing (feel free to add to the list if I'm missing anything):

* Cache: we disable disk and offline caches, and continue using the memory cache.  The memory cache may contain items specific to user's identity (for example, a page sent by a web application which is dynamically generated using user's cookie, and has stayed in the cache).  Therefore, we may want to clear the memory cache before proceeding in the private browsing mode as well.  The current implementation empties the memory cache before leaving the private browsing mode, so that no content from the private session lives in the memory after it.

* History: we simply disable adding new history items in the private browsing mode.  The previously existing history items are usable though.  The URLs stored in the history may contain session IDs, etc. (think of sites which embed session IDs in URLs instead of cookies), but disabling history completely does not seem like a good option.

* Form AutoComplete: we disable adding new form autocomplete items in private browsing mode, but using previously stored items is allowed.  I can't think of why this may be a possible privacy concern...

* Download Manager: we disable keeping downloaded items in the downloads history in the private browsing mode, but old items remain there, which looks innocent.

* Cookies: we currently allow cookies to be saved in the private browsing mode, and delete them once we're leaving this mode.  The entire collection of cookies is usable in private browsing mode.  This implementation is not good (see the issues in comment 21), but I have not yet worked on this part at all (I'm waiting on bug 411952...).  The implementation you suggest in comment 56 actually seems good enough: access to the existing cookies should be blocked, but new cookies should be able to be saved to and restored from memory while in private browsing mode.  Exiting this mode will delete the in-memory temporary cookies table, and will switch to the default set of cookies for use in normal browsing mode.  The implementation of such behavior is more difficult than the existing plan, but it may be worth it (I'm OK with the implementation challenge.)

* Session Store: the session store in private browsing mode is prevented from writing anything to disk, but maintains its own memory data collections.  Upon leaving this mode, the entire data collected throughout this mode is dumped, and the data is collected once again.  Upon entering the private browsing mode, the previously collected session store data is available (for example, you can undo closed tabs), which is no bigger a privacy risk than the previously stored history items being accessible.  (BTW, now that I think of it, Session Store should restore its exact data from the last snapshot before entering the private browsing mode, and then scan for newly added/changed tabs and windows.  This way the recently closed tabs at the start of the private browsing session are available after exiting it.  I'll implement this in a future version of the patch.)

* Password Manager: the password manager should not prompt to save new login info in the private browsing mode, but should be able to autofill form elements with previously stored info.  This is exactly like the Form History and I think it's OK.

* Permissions Manager: the permissions manager should be prevented from storing new permissions in the private browsing mode (because storing new permissions could reveal sites visited in the private browsing mode.)  Respecting previously stored permissions should be OK (like the case of Form History).  BTW I'm currently working on this, and I'll post a patch shortly.

* Certificate Manager: the certificate exceptions should not be stored in the private browsing mode (like the case for Permissions Manager).  Respecting previously stored exceptions should be OK (like the case of Permissions Manager).

* Add-ons Install Whitelist: The same goes here like Permissions Manager and Certificate Manager.  (BTW, wasn't add-ons install whitelist supposed to be eliminated in Firefox 3?  What is the status of it?  Should I invest time on working on it in this bug?)

* Error Console: the error console should be prevented from accepting messages while in private browsing mode, because many messages contain site URLs.  Viewing the previous messages from normal browsing modes should not be a privacy concern.

Comments/suggestions/criticisms are most welcome!
This patch implements the backend changes in the nsPermissionManager, and also disabled the UI elements responsible for changing permissions in the browser code.  With this patch, when the user enters the private browsing mode, she cannot set any permissions on that site (because otherwise the fact that she has visited that site gets exposed later), but can consume the existing permissions.

The UI elements handled include:
* Firefox preferences window (the Exceptions buttons for images, cookies, and installs)
* Firefox notification box menu on pages with a popup
* Firefox permission editing controls in the Page Info dialog
Attachment #296686 - Attachment is obsolete: true
Here's an idea with cookies that I think might be ideal:
Copy the existing cookie table to a new table, like moz_old_cookies (better name needed).
Continue to use the existing cookie table, but once private browsing mode is exited, we copy the old table back into the new one and delete moz_old_cookies.

This means we get *all* of the cookies we had before when we enter, and then we go back to the exact state we were in before once we exit.

It might also be better to just copy the db file (probably faster).
(In reply to comment #59)
> Here's an idea with cookies that I think might be ideal:

sdwilsh speaketh the truth. there are a few problems with your approach (patch 6):

1) if you use nsCookie::LastAccessed(), you're going to end up deleting cookies that were accessed by (i.e., sent back to) a webserver, but never modified. so you'll be overeager in your deletion.

2) if you use the cookie's creation time (approximately nsCookie::CreationID()), you'll catch cookies created since the private browsing epoch, but not cookies that were modified. creation times are not updated when a cookie's value is changed, which is important here. in addition, you'll allow last accessed times to be updated on sites that were visited, which (although pretty minor) would allow someone to tell what those sites were. (we don't currently expose that information on nsICookie{2}, but someone could sniff it from the db.)

3) i'm not sure i like the implementation of private browsing being pushed into the cookie/permission services, since these are part of the core netwerk module and are supposed to be as little app-specific as possible - but if it's well-written and minimal, i may be convinced.

a better approach would be to back up the original cookie and db files to a known location, do the private browsing, and then restore them. you'll have to figure out what to do in event of crash, though - we don't want that private data being left around, or orphaned.
Comment on attachment 296738 [details] [diff] [review]
WIP Patch 6 - nsPermissionManager


>+            var prefBranch = Cc["@mozilla.org/preferences-service;1"].
>+                             getService(Ci.nsIPrefService).getBranch("privacy.");
>+            in_pb = prefBranch.getBoolPref("privatebrowsing");

Hmm, I wonder if this might be better implemented as an observer+topic.

Same net effect, just a conceptual difference.


>     promptAuth : function (aChannel, aLevel, aAuthInfo) {
...
>-            var notifyBox = this._getNotifyBox();
>-            if (notifyBox)
>-                this._removeSaveLoginNotification(notifyBox);
>+            var notifyBox = null;
>+            if (!inPrivateBrowsing) {
>+                var notifyBox = this._getNotifyBox();
>+                if (notifyBox)
>+                    this._removeSaveLoginNotification(notifyBox);
>+            }

This shouldn't change. If we're about to prompt for a login, any existing save-password notification bar should be removed to avoid confusion about what's being saved.


>             var canRememberLogin = this._pwmgr.getLoginSavingEnabled(hostname);
>-        
>+
>             // if checkboxLabel is null, the checkbox won't be shown at all.
>-            if (canRememberLogin && !notifyBox)
>+            if (canRememberLogin && !notifyBox && !inPrivateBrowsing)
>                 checkboxLabel = this._getLocalizedString("rememberPassword");

I think the logic here would be simpler if canRememberLogin was forced to |false| is private browsing mode is enabled.
Comment on attachment 296738 [details] [diff] [review]
WIP Patch 6 - nsPermissionManager

>Index: toolkit/components/downloads/src/nsDownloadManager.cpp
>+  // If the user is in private browsing mode, delete the download record when
>+  // completed
>+  PRBool privMode = PR_FALSE;
>+  pref->GetBoolPref(PREF_PRIVATEBROWSING, &privMode);
>+  if (privMode)
>+    return 0;
>+
please grab the result of GetBoolPref, and in the if statement, check if it was successful as well.
(In reply to comment #59)
> Here's an idea with cookies that I think might be ideal:
> Copy the existing cookie table to a new table, like moz_old_cookies (better
> name needed).
> Continue to use the existing cookie table, but once private browsing mode is
> exited, we copy the old table back into the new one and delete moz_old_cookies.
> 
> This means we get *all* of the cookies we had before when we enter, and then we
> go back to the exact state we were in before once we exit.
> 
> It might also be better to just copy the db file (probably faster).

See comment 32:

"By choosing to write *some* data to disk (perhaps in an encrypted format) we
have broken a clear and easy to understand contract between Firefox and the
user. The user / security expert will not be sure that there is no security
risk."

Mconnor: what do you think?
(In reply to comment #60)
> (In reply to comment #59)
> > Here's an idea with cookies that I think might be ideal:
> 
> sdwilsh speaketh the truth. there are a few problems with your approach (patch
> 6):

Hmm, I haven't actually touched the cookie code in any significant way from Michael's original patch yet...

> 1) if you use nsCookie::LastAccessed(), you're going to end up deleting cookies
> that were accessed by (i.e., sent back to) a webserver, but never modified. so
> you'll be overeager in your deletion.

Yeah, and we don't want that.

> 2) if you use the cookie's creation time (approximately
> nsCookie::CreationID()), you'll catch cookies created since the private
> browsing epoch, but not cookies that were modified. creation times are not
> updated when a cookie's value is changed, which is important here. in addition,
> you'll allow last accessed times to be updated on sites that were visited,
> which (although pretty minor) would allow someone to tell what those sites
> were. (we don't currently expose that information on nsICookie{2}, but someone
> could sniff it from the db.)

Agreed.

> 3) i'm not sure i like the implementation of private browsing being pushed into
> the cookie/permission services, since these are part of the core netwerk module
> and are supposed to be as little app-specific as possible - but if it's
> well-written and minimal, i may be convinced.

Hmm, what do you think specifically about the implementation in the cache and permissions backends in patch 6 (attachment 296738 [details] [diff] [review])?

Based on the service which needs behavior modifications, there may be things which can be done from the UI.  For example, for the permissions service, my patch changes the nsPermissionManager slightly, and also disables any place in the UI which is used to add modify anything in nsPermissionManager.  But I'm not sure how a similar "higher-level" approach can be taken for cache and cookie services, since they are accessed by other modules in netwerk, and patching the UI won't work in those cases (at least, I can't see how it would work.)

> a better approach would be to back up the original cookie and db files to a
> known location, do the private browsing, and then restore them. you'll have to
> figure out what to do in event of crash, though - we don't want that private
> data being left around, or orphaned.

Hmm, IINM, as long as we write the cookies on disk (no matter in the original or in a new db file), we are risking the information to get leaked.  There is not much we can do in case of a crash, and there is even less that can be done in case Firefox is "end-tasked" without a chance to do any cleanup.  And by using sqlite as the backend, we ensure that the data is left in a usable state on the disk, so we can't even hope for it to get corrupted.  :-)

This is why I think an approach such as comment 56 is better in this regard (I'm talking about the having an in-memory DB part, not necessarily avoiding using the existing cookies.)
(In reply to comment #62)
> (From update of attachment 296738 [details] [diff] [review])
> >Index: toolkit/components/downloads/src/nsDownloadManager.cpp
> >+  // If the user is in private browsing mode, delete the download record when
> >+  // completed
> >+  PRBool privMode = PR_FALSE;
> >+  pref->GetBoolPref(PREF_PRIVATEBROWSING, &privMode);
> >+  if (privMode)
> >+    return 0;
> >+
> please grab the result of GetBoolPref, and in the if statement, check if it was
> successful as well.

The idea here (and in other parts of the patch) is to assume that we are not in the safe browsing mode, and try to prove otherwise.  That is, if the GetBoolPref fails for any reason, privMode would remain PR_FALSE...  Is this assumption correct?  If not, then I would have change this (and possibly other places in the patch as well).
Thanks for the comments, Justin.

(In reply to comment #61)
> (From update of attachment 296738 [details] [diff] [review])
> 
> >+            var prefBranch = Cc["@mozilla.org/preferences-service;1"].
> >+                             getService(Ci.nsIPrefService).getBranch("privacy.");
> >+            in_pb = prefBranch.getBoolPref("privatebrowsing");
> 
> Hmm, I wonder if this might be better implemented as an observer+topic.
> 
> Same net effect, just a conceptual difference.

See the first part of my comments in comment 32.  If I'm missing something here, I'd change the patch accordingly, but no one has commented on my reasoning yet...

> >     promptAuth : function (aChannel, aLevel, aAuthInfo) {
> ...
> >-            var notifyBox = this._getNotifyBox();
> >-            if (notifyBox)
> >-                this._removeSaveLoginNotification(notifyBox);
> >+            var notifyBox = null;
> >+            if (!inPrivateBrowsing) {
> >+                var notifyBox = this._getNotifyBox();
> >+                if (notifyBox)
> >+                    this._removeSaveLoginNotification(notifyBox);
> >+            }
> 
> This shouldn't change. If we're about to prompt for a login, any existing
> save-password notification bar should be removed to avoid confusion about
> what's being saved.

You're right, sorry for the mistake.

> >             var canRememberLogin = this._pwmgr.getLoginSavingEnabled(hostname);
> >-        
> >+
> >             // if checkboxLabel is null, the checkbox won't be shown at all.
> >-            if (canRememberLogin && !notifyBox)
> >+            if (canRememberLogin && !notifyBox && !inPrivateBrowsing)
> >                 checkboxLabel = this._getLocalizedString("rememberPassword");
> 
> I think the logic here would be simpler if canRememberLogin was forced to
> |false| is private browsing mode is enabled.

Done.

The new patch fixes the above two issues, and also adds browser/components/preferences/permissions.js which was accidentally dropped off from patch 6.
Attachment #296738 - Attachment is obsolete: true
(In reply to comment #63)
> This is why I think an approach such as comment 56 is better in this regard
> (I'm talking about the having an in-memory DB part, not necessarily avoiding
> using the existing cookies.)

hmm. in fact - you can avoid that problem by closing the db connection in the cookieservice, making further operations occur in-memory. on exiting private browsing mode, just reload the cookie table from disk (i.e. call InitDB()).

that should be very simple to implement - and requires only a few lines of code in nsCookieService::Observe(). for permissionmanager, you could do something similar.
(In reply to comment #67)
> hmm. in fact - you can avoid that problem by closing the db connection in the
> cookieservice, making further operations occur in-memory. on exiting private
> browsing mode, just reload the cookie table from disk (i.e. call InitDB()).

Wow, I didn't know the cookie service works in this way.  So, if the DB connection is unavailable, will all operations (adding cookies, updating cookies, etc.) happen successfully in memory?  And will the full list of cookies read from the DB upon initialization remain intact (so that old cookies do not get lost when entering the private browsing mode)?

If the above is true, then it will indeed be very simple to implement.

> that should be very simple to implement - and requires only a few lines of code
> in nsCookieService::Observe(). for permissionmanager, you could do something
> similar.

Does the above apply to the permission manager as well?  Can I safely close the DB connection in the middle of its operation, and all ops from that point on get executed in memory, and later on when a new DB connection is opened, everything will revert back to just like it was before closing the connection?
(In reply to comment #68)
> Wow, I didn't know the cookie service works in this way.  So, if the DB
> connection is unavailable, will all operations (adding cookies, updating
> cookies, etc.) happen successfully in memory?  And will the full list of
> cookies read from the DB upon initialization remain intact (so that old cookies
> do not get lost when entering the private browsing mode)?

indeed! the cookie service effectively has two data stores - on-disk and in-memory. on loading a profile, it reads data from the on-disk db into memory, and from that point on keeps them synced. (note that session cookies are held only in memory, and never see the platter - so the in-memory db is a superset of the on-disk one.)

simply closing that db connection at any point in time will cause it to run only from memory. it's designed that way - having a profile around to access is purely optional (think embeddors here).

> Does the above apply to the permission manager as well?  Can I safely close the
> DB connection in the middle of its operation, and all ops from that point on
> get executed in memory, and later on when a new DB connection is opened,
> everything will revert back to just like it was before closing the connection?

permission manager works identically. although by "opening a new connection" you're really clearing the existing in-memory table, and then calling InitDB() to reload the on-disk data.
(In reply to comment #69)
> indeed! the cookie service effectively has two data stores - on-disk and
> in-memory. on loading a profile, it reads data from the on-disk db into memory,
> and from that point on keeps them synced. (note that session cookies are held
> only in memory, and never see the platter - so the in-memory db is a superset
> of the on-disk one.)
> 
> simply closing that db connection at any point in time will cause it to run
> only from memory. it's designed that way - having a profile around to access is
> purely optional (think embeddors here).

OK, I'm amused!  :-)  Thanks a lot for the pointer, Dan.  So, I guess this removes the dependency on bug 411952...

The only thing which needs to be decided before I can come up with a patch is if we want to keep the previously stored cookies around, or start from scratch as Alex suggested in comment 56.  Faaborg, Beltzner, Mconnor: any ideas?

> permission manager works identically. although by "opening a new connection"
> you're really clearing the existing in-memory table, and then calling InitDB()
> to reload the on-disk data.

Yeah, that is what we would want.  This makes the whole UI changes to disable setting new permissions introduced in patch 6 redundant as well: users *will* be able to set new permissions in the private browsing mode, only those new permissions would last until the end of their private browsing session, and would get vanished right after they exit the private browsing session.

BTW, this makes me think that the private browsing should not last between browser session restarts.  Think of what would happen to the cookies if the user closes the browser without exiting the private browsing mode...  They would get lost (because they are stored in memory.)  I think this gives me a clear idea why private browsing mode should not be retained after starting the browser.  If we have an indicator in the primary UI (like Alex suggested in comment 41) then this won't be much of an issue, because the user will know immediately that she's no longer in the private browsing mode after she restarts the browser.  And it makes the observer/notification implementation suggested by mconnor and Justin before viable.  We'll probably want a component which runs as a service which could be queried for the state of the private browsing mode, and can be used to initiate the browsing mode.  This component can be used for all higher level code (possibly excluding the netwerk/permission code, which can use raw observers.)

This sounds like quite a bit of work, but I'll try to come up with a new patch shortly.
(In reply to comment #69)
> indeed! the cookie service effectively has two data stores - on-disk and
> in-memory. on loading a profile, it reads data from the on-disk db into memory,
> and from that point on keeps them synced. (note that session cookies are held
> only in memory, and never see the platter - so the in-memory db is a superset
> of the on-disk one.)

One more question here.  Would closing the connection automatically lead to the in-memory table to clear, or not?  If not, is there an easy way to clear it if needed?  This would be important when entering the private browsing mode.

Also, like what you mentioned about the permission manager, would re-opening the connection cause the in-memory table to be reloaded from the DB?  This would be important when exiting the private browsing mode.
No longer depends on: 411952
(In reply to comment #70)
> Yeah, that is what we would want.  This makes the whole UI changes to disable
> setting new permissions introduced in patch 6 redundant as well: users *will*
> be able to set new permissions in the private browsing mode, only those new
> permissions would last until the end of their private browsing session, and
> would get vanished right after they exit the private browsing session.

i like this approach much better!

(In reply to comment #71)
> One more question here.  Would closing the connection automatically lead to the
> in-memory table to clear, or not?  If not, is there an easy way to clear it if
> needed?  This would be important when entering the private browsing mode.

it won't - see {nsCookieService,nsPermissionManager}::RemoveAllFromMemory(). i'd be in favor of not doing this, though the UX guys might think it's best so that users have the "feel" of things being reset/private.

closing the db connection is as simple as nulling out mDBConn.

> Also, like what you mentioned about the permission manager, would re-opening
> the connection cause the in-memory table to be reloaded from the DB?  This
> would be important when exiting the private browsing mode.

it won't ;) you'll have to call {nsCookieService,nsPermissionManager}::InitDB().

one consideration here, specific to cookies: say the user starts the browser (loading the cookie db from disk), and then browses around for a while (accumulating session cookies in memory, that never get written to disk). they now have both persistent and session cookies in memory. then they enter PB mode, and accumulate more cookies. on exiting that mode, we throw away everything and reload persistent cookies from disk. those session cookies they acquired before entering PB mode are now lost. (also consider the case where the user has 'limit cookie lifetime to session' enabled, so all their cookies are session-only.) do we care about these cases? phrased another way, is it acceptable for exiting PB mode to behave like a browser restart?
(In reply to comment #72)
> (In reply to comment #70)
> > Yeah, that is what we would want.  This makes the whole UI changes to disable
> > setting new permissions introduced in patch 6 redundant as well: users *will*
> > be able to set new permissions in the private browsing mode, only those new
> > permissions would last until the end of their private browsing session, and
> > would get vanished right after they exit the private browsing session.
> 
> i like this approach much better!

Me too.  :-)

> (In reply to comment #71)
> > One more question here.  Would closing the connection automatically lead to the
> > in-memory table to clear, or not?  If not, is there an easy way to clear it if
> > needed?  This would be important when entering the private browsing mode.
> 
> it won't - see {nsCookieService,nsPermissionManager}::RemoveAllFromMemory().
> i'd be in favor of not doing this, though the UX guys might think it's best so
> that users have the "feel" of things being reset/private.
> 
> closing the db connection is as simple as nulling out mDBConn.

Thanks for the tips!

> > Also, like what you mentioned about the permission manager, would re-opening
> > the connection cause the in-memory table to be reloaded from the DB?  This
> > would be important when exiting the private browsing mode.
> 
> it won't ;) you'll have to call
> {nsCookieService,nsPermissionManager}::InitDB().

Hmm, nsCookieService::InitDB() calls nsCookieService::Read() which leads to items in the DB be read into the memory table, right?

> one consideration here, specific to cookies: say the user starts the browser
> (loading the cookie db from disk), and then browses around for a while
> (accumulating session cookies in memory, that never get written to disk). they
> now have both persistent and session cookies in memory. then they enter PB
> mode, and accumulate more cookies. on exiting that mode, we throw away
> everything and reload persistent cookies from disk. those session cookies they
> acquired before entering PB mode are now lost. (also consider the case where
> the user has 'limit cookie lifetime to session' enabled, so all their cookies
> are session-only.) do we care about these cases? phrased another way, is it
> acceptable for exiting PB mode to behave like a browser restart?

Hmm, would it be possible to get a copy of the in-memory hash table of cookies, and restore it when exiting from the private browsing mode, instread of calling nsCookieService::RemoveAllFromMemory()?  This way, everything would get restored to what it was exactly before initiating the private browsing session.
(In reply to comment #73)
> Hmm, nsCookieService::InitDB() calls nsCookieService::Read() which leads to
> items in the DB be read into the memory table, right?

right.

> Hmm, would it be possible to get a copy of the in-memory hash table of cookies,
> and restore it when exiting from the private browsing mode, instread of calling
> nsCookieService::RemoveAllFromMemory()?  This way, everything would get
> restored to what it was exactly before initiating the private browsing session.

yes, though that'd be a little more complicated. we should probably decide how we want this to behave before embarking on that ;)
(In reply to comment #65)
> The idea here (and in other parts of the patch) is to assume that we are not in
> the safe browsing mode, and try to prove otherwise.  That is, if the
> GetBoolPref fails for any reason, privMode would remain PR_FALSE...  Is this
> assumption correct?  If not, then I would have change this (and possibly other
> places in the patch as well).
XPCOM rules say you should not trust an out parameter if the method did not return successfully.
(In reply to comment #74)
> (In reply to comment #73)
> > Hmm, nsCookieService::InitDB() calls nsCookieService::Read() which leads to
> > items in the DB be read into the memory table, right?
> 
> right.
> 
> > Hmm, would it be possible to get a copy of the in-memory hash table of cookies,
> > and restore it when exiting from the private browsing mode, instread of calling
> > nsCookieService::RemoveAllFromMemory()?  This way, everything would get
> > restored to what it was exactly before initiating the private browsing session.
> 
> yes, though that'd be a little more complicated. we should probably decide how
> we want this to behave before embarking on that ;)

OK, waiting for comments from UX guys...
(In reply to comment #75)
> XPCOM rules say you should not trust an out parameter if the method did not
> return successfully.

Oh, thanks for mentioning this.  I'll post an updated patch shortly.
> > Hmm, would it be possible to get a copy of the in-memory hash table of cookies,
> > and restore it when exiting

>OK, waiting for comments from UX guys...

To make sure I fully understand the question, what are the interface implications of restoring the in-memory cookies from before entering private browsing mode.  Would this enable use to let the user enter and exit without having to end their session?
>The only thing which needs to be decided before I can come up with a patch is
>if we want to keep the previously stored cookies around, or start from scratch
>as Alex suggested in comment 56.  Faaborg, Beltzner, Mconnor: any ideas?

It would be good if beltzner or mconnor weighed in as well, but after thinking about this some more, I believe users will be surprised if sites recognize them after entering private browsing mode.  If you think of private browsing mode as putting on a mask, than the first thing that happens is Amazon.com says "Hello Alex Faaborg!" that implies the mask isn't very good.
(In reply to comment #78)
> >OK, waiting for comments from UX guys...
> 
> To make sure I fully understand the question, what are the interface
> implications of restoring the in-memory cookies from before entering private
> browsing mode.  Would this enable use to let the user enter and exit without
> having to end their session?

There are two options to implement private browsing -- cookie-wise:

1. Give the user a "clean" session by not using any previously used cookies whatsoever.  This is just like the case where they create a new profile and use it to browse the web.  The cookies can be stored during user's private browsing session.  After the user exits, any cookies stored will be discarded.  All cookies are stored only in memory, so no track of user's actions are saved to disk.

2. Just like case 1, with the exception that the previously stored cookies are not discarded; they're used as before, but no update on the cookies will be stored on disk, and the exact snapshot of the in-memory cookie table before entering the private browsing mode would be available afterward, as if no private browsing has ever occurred.
(In reply to comment #79)
> It would be good if beltzner or mconnor weighed in as well, but after thinking
> about this some more, I believe users will be surprised if sites recognize them
> after entering private browsing mode.  If you think of private browsing mode as
> putting on a mask, than the first thing that happens is Amazon.com says "Hello
> Alex Faaborg!" that implies the mask isn't very good.

Or Amazon is too good!  ;-)

Seriously, the more I think about it, the more logical your approach seems to me...
Here is a new version of the patch, that I've rewritten from scratch.  This version replaces the "privacy.privatebrowsing" pref with the "browser:private-browsing" notification.  This notification gets sent by the gPrivateBrowsingMgr object defined in browser.js.  The data value of this notification determines if we're entering the private browsing mode or leaving it.  I have added a quick and dirty item to the Tools menu to toggle the private browsing mode to aid testing, and this will be removed in the final version of the patch, as the UI for private browsing mode would be implemented in bug 411929.

This patch implements the changes to the cache, history, form fill history and download manager components.  The changes to the cookies, session store, login manager, permission manager, and error console components will be implemented shortly in a future WIP patch.  For now, I'm assuming that we want the cookie behavior that Alex suggested earlier, unless I hear otherwise from him or mconnor/beltzner.
Attachment #296798 - Attachment is obsolete: true
Attachment #297698 - Attachment description: WIP Patch 7 - Rewrite using notification/observers (not completed yet) → WIP Patch 8 - Rewrite using notification/observers (not completed yet)
Sorry to CC you again, Simon.

I was wondering if it's possible to restore the SessionStore service after the private browsing mode in a way that the SessionStore data for recently closed tabs is exactly like what it was right before entering the private browsing mode.

Here's the use case:
1) John is browsing the web.  His recently closed tabs list includes [Google, Gmail, CNN].
2) He enters private browsing mode, opens a new tab to browse Amazon, and he closes it later on.  His recently closed tabs list at this point: [Google, Gmail, CNN, Amazon].
3) He exits private browsing mode.

With the current implementation, after 3 the recently closed tabs list is [].  I like to restore it to [Google, Gmail, CNN], but I'm not sure how to go about it and not hurt other parts of the code...

Suggestions welcome.  Thanks!
(In reply to comment #75)
> (In reply to comment #65)
> > The idea here (and in other parts of the patch) is to assume that we are not in
> > the safe browsing mode, and try to prove otherwise.  That is, if the
> > GetBoolPref fails for any reason, privMode would remain PR_FALSE...  Is this
> > assumption correct?  If not, then I would have change this (and possibly other
> > places in the patch as well).
> XPCOM rules say you should not trust an out parameter if the method did not
> return successfully.

BTW, since I'm no longer using a pref, this should no longer be relevant.
(In reply to comment #83)
> in a way that the SessionStore data for recently closed tabs is exactly
> like what it was right before entering the private browsing mode.

When entering private browsing mode, iterate through all _windows and for each of them clone the _closedTabs array into _closedTabsBackup (where cloning is best achieved through |eval(closedTabs.toSource())|). Then you can just move that backup back when you exit private browsing mode and delete _closedTabsBackup (and make sure that _closedTabs is still no larger than what .max_tabs_undo dictates).
Attached patch WIP Patch 9 - Nearly complete (obsolete) — Splinter Review
This patch is a followup to attachment 297698 [details] [diff] [review].

What's new:

* Cookie service: the implementation follows Dan's hint in comment 67, and Alex's idea in comment 54.  The in-memory DB would be restored when exiting the private browsing mode, which addresses an issue Dan raised in comment 72.

* Permission manager: the implementation has been changed to follow Dan's hint in comment 67.

* Login prompter: the implementation uses a separate object with greater lifetime than that of prompter objects, so that the true state of private browsing can be obtained at any time.

* Content prefs: no prefs will be saved during the private browsing mode, which makes it impossible to track user actions in private browsing mode based on the content prefs DB.

* Session store: added the implementation Simon mentioned in comment 85.  Note: this does not work as intended (i.e., the Recently closed tabs menu item is grayed out after exiting the private browsing mode).  Need to investigate this further.

* Console service: added a simple mechanism to pause/resume recording of messages in the console service, and modified Private Browsing Manager in browser.js to use this facility to make sure nothing is logged to the console service during the private browsing session.  I didn't modify the service itself to observe browser:private-browsing notifications because that could lead to circular calls made back to the service by the observer service.

* Private Browsing manager: handled quit-application-granted, to make sure to exit private browsing mode just before application quit, so that services which need cleanup can get the proper exit notification.

The only other service which needs modification is the certificate exceptions list.  I'll implement that in a next patch, and will ask for review on various parts of the patch then.  If anyone can think of other services which need modification for private browsing mode, please let me know.
Attachment #297698 - Attachment is obsolete: true
Attached patch WIP Patch 9 - Nearly complete (obsolete) — Splinter Review
Attachment 297997 [details] [diff], updated to trunk.
Attachment #297997 - Attachment is obsolete: true
(In reply to comment #86)
> * Session store: added the implementation Simon mentioned in comment 85.

I'll review the changes when you're done. Just a note on your issue: it doesn't work because of the splicing in _restoreRecentlyClosedTabs. Instead you'll have to |.slice(0, maxTabsUndo)|...

> * Console service: added a simple mechanism to pause/resume recording of
> messages in the console service

I'd rather see an attribute "paused" (or "logging", "enabled" or something alike) than two methods "pause" and "resume" so that e.g. Firebug can easily determine whether it should inform the user that she shouldn't be expecting any errors in the console (without having to keep track of private browsing mode).

> * Private Browsing manager: handled quit-application-granted, to make sure to
> exit private browsing mode just before application quit

That code will have to move into an XPCOM component (e.g. nsBrowserGlue) or it'll fail to run when the last closed window isn't a browser one.
(In reply to comment #88)
> (In reply to comment #86)
> > * Session store: added the implementation Simon mentioned in comment 85.
> 
> I'll review the changes when you're done. Just a note on your issue: it doesn't
> work because of the splicing in _restoreRecentlyClosedTabs. Instead you'll have
> to |.slice(0, maxTabsUndo)|...

I should be ashamed of not catching this myself.  Of course you were absolutely right!  Fixed.

> > * Console service: added a simple mechanism to pause/resume recording of
> > messages in the console service
> 
> I'd rather see an attribute "paused" (or "logging", "enabled" or something
> alike) than two methods "pause" and "resume" so that e.g. Firebug can easily
> determine whether it should inform the user that she shouldn't be expecting any
> errors in the console (without having to keep track of private browsing mode).

Done.  The new patch uses the |paused| attribute on the console service.

> > * Private Browsing manager: handled quit-application-granted, to make sure to
> > exit private browsing mode just before application quit
> 
> That code will have to move into an XPCOM component (e.g. nsBrowserGlue) or
> it'll fail to run when the last closed window isn't a browser one.

I moved the whole stuff in browser.js to its own XPCOM component in browser/components.
Attachment #298002 - Attachment is obsolete: true
Ehsan - there's a lot of great work in here. Any chance of you being able to put this all together in an XPI to get broader testing of the functionality?
(In reply to comment #90)
> Ehsan - there's a lot of great work in here. Any chance of you being able to
> put this all together in an XPI to get broader testing of the functionality?

I'd love to, but I'm not sure how to do it, since the code touches a lot of core components, such as cache, cookies, permissions, etc.  Maybe it would be better to provide builds with this patch applied for users to test?  I can provide Windows and Linux builds, but I need help in creating a Mac OS X build because I don't have access to any Mac computer...

Let me know if providing build would serve this purpose.
(In reply to comment #91)
> but I need help in creating a Mac OS X build

"buildbot try" can roll builds on all the major plats for you, and make them available for download - check out build.mozilla.org when you have your cvs account set up. in the meantime, if you attach a patch here, someone can probably do it for you ;)
(In reply to comment #92)
> "buildbot try" can roll builds on all the major plats for you, and make them
> available for download - check out build.mozilla.org when you have your cvs
> account set up. in the meantime, if you attach a patch here, someone can
> probably do it for you ;)

Thanks for the tip.  I guess I'll ask someone to make the builds on this bug as soon as I'm finished with the patch here.  I'm currently stuck by some bug in the cert override service, which I'm going to file shortly... :(
Having unit tests would also help integration I guess. I was thinking of
something like this:
Setup a set of pages running on httpd.js for testing cookies, auto-completion,
form entry and others. Then develop a browser chrome test
(http://developer.mozilla.org/en/docs/Browser_chrome_tests) which will simulate
navigation and use features affecting the profile state (browse to pages
setting cookies, simulate a password entry, simulate downloading a file, ...).

This simulated session could be run a first time with no private browsing mode
and check that all works well (cookies saved, auto-completion saved, ...). Then
we sanitize the profile, toggle private browsing mode and run the navigation
simulation again and the toggle private browsing mode off. After that, we check
that nothing is saved in the profile.

I can imagine two ways of checking that nothing is saved in the profile while
in private browsing mode:
1) one high level check that would ensure that no files were written during
private private browsing mode by checking file last write time (hoping this
works in a cross platform way). There could be a white list for things that are
allowed to change and don't introduce a privacy risk (like
urlclassifier3.sqlite I guess). The advantage of this kind of test is that it
could detect new features added without them managing the private browsing
mode, preventing privacy leaks added in the future.
2) Service specific checks (checking cookies, cache, form history, ...)

I don't think something similar already exist for the sanitizer service (clear
private data). Such a framework if developed could then test both the private
browsing mode and the sanitizer. For testing the sanitizer, we launch the
simulated browsing, run the sanitizer and check that nothing is saved in test 2
above (test 1 would not be effective, as files are written during navigation).

That was some ideas I had for making private browsing mode more robust. Of
course it's certainly more easy to write down than to implement ;-)


Maybe another service that could participate in the private browsing mode is
the site specific preferences? If you navigate to a site and change the zoom
level this information is saved in this store, thus revealing you visited that
site. This could use the same handling as the cookies: site specific
preferences are used as normal while in private browsing mode, but the store is
not updated in this mode (this means your zoom settings are not reset in
private browsing mode for instance).
Depends on: 413627
Thanks for your input, Sylvian.

(In reply to comment #94)
> Having unit tests would also help integration I guess. I was thinking of
> something like this:
> Setup a set of pages running on httpd.js for testing cookies, auto-completion,
> form entry and others. Then develop a browser chrome test
> (http://developer.mozilla.org/en/docs/Browser_chrome_tests) which will simulate
> navigation and use features affecting the profile state (browse to pages
> setting cookies, simulate a password entry, simulate downloading a file, ...).

Hmmm good idea.  I'm totally new in writing chrome tests, though.  I'm going through the docs on MDC right now, but if anyone wants to jump in and help on this, I'd be more than happy to accept inputs/hints/written tests.  :-)

> I don't think something similar already exist for the sanitizer service (clear
> private data). Such a framework if developed could then test both the private
> browsing mode and the sanitizer. For testing the sanitizer, we launch the
> simulated browsing, run the sanitizer and check that nothing is saved in test 2
> above (test 1 would not be effective, as files are written during navigation).

That would be part of another bug, right?  Maybe you could file one to keep it on-track?

> Maybe another service that could participate in the private browsing mode is
> the site specific preferences? If you navigate to a site and change the zoom
> level this information is saved in this store, thus revealing you visited that
> site.

I added this component in the WIP Patch 9 (attachment 297997 [details] [diff] [review]).

> This could use the same handling as the cookies: site specific
> preferences are used as normal while in private browsing mode, but the store is
> not updated in this mode (this means your zoom settings are not reset in
> private browsing mode for instance).

This implementation is actually quite complicated.  It was easy in case of cookies and the permission manager because they already provided an in-memory DB which is a superset of the on-disk DB.  The contentprefs service, however, does not even perform in-memory caching, and it reads and writes all the prefs directly from/to the DB.  And we have made similar tradeoffs in other services.  For example, the pages visited in the private browsing mode don't end up in your history even when you have not left the private browsing mode.  I think a similar tradeoff here would be acceptable, wouldn't it?
(In reply to comment #95)
> 
> That would be part of another bug, right?  Maybe you could file one to keep it
> on-track?

Yes, I think so. I'll do some research if it is filed already and enter a new one if needed.

> I added this component in the WIP Patch 9 (attachment 297997 [details] [diff] [review]).

Sorry I didn't see it while skimming through the patch.

> This implementation is actually quite complicated.  It was easy in case of
> cookies and the permission manager because they already provided an in-memory
> DB which is a superset of the on-disk DB.  The contentprefs service, however,
> does not even perform in-memory caching, and it reads and writes all the prefs
> directly from/to the DB.  And we have made similar tradeoffs in other services.
>  For example, the pages visited in the private browsing mode don't end up in
> your history even when you have not left the private browsing mode.  I think a
> similar tradeoff here would be acceptable, wouldn't it?

Seems acceptable to me. If I understand the patch well, the content preferences can be read but not written in private browsing mode. This means a site where you raised the zoom level will still be zoomed in private browsing mode as expected.

(In reply to comment #96)
> (In reply to comment #95)
> > 
> > That would be part of another bug, right?  Maybe you could file one to keep it
> > on-track?
> 
> Yes, I think so. I'll do some research if it is filed already and enter a new
> one if needed.

Thanks!  Please CC me if you find an existing one (or file a new bug).

> Seems acceptable to me. If I understand the patch well, the content preferences
> can be read but not written in private browsing mode. This means a site where
> you raised the zoom level will still be zoomed in private browsing mode as
> expected.

Yes, exactly.
This new patch makes nsCertOverrideService honor private browsing mode.  The approach used is to store the overrides in memory while we are in private browsing mode, and clear them up after exiting this mode.  Nothing will be written to disk, even if the override entry is set to permanent mode.

Note that I have not been able to test this because of bug 413627.  I'll wait for some time and try to figure out a solution.  Then I'll ask for review on this patch.  Please let me know if you know how to solve this problem.  For more info, see bug 413627 comment 0.
Attachment #298145 - Attachment is obsolete: true
BTW, according to comment 92, if anyone can help build a test version for all the three platforms, that would be great.  Please use the WIP 11 patch for this purpose.
Oops, I forgot to mention that I missed browser/components/Makefile.in in my WIP 9 and 10 patches, which would cause the privatebrowsing XPCOM component not to get built.  This is fixed in WIP 11 as well.
I gave the WIP 11 patch to the try-server, so a private browsing-build for all three platforms should appear in about 1 hour and 30 minutes here: https://build.mozilla.org/tryserver-builds/
(In reply to comment #97)
> Thanks!  Please CC me if you find an existing one (or file a new bug).

I filed bug 413659 for this.

(In reply to comment #101)
> I gave the WIP 11 patch to the try-server, so a private browsing-build for all
> three platforms should appear in about 1 hour and 30 minutes here:
> https://build.mozilla.org/tryserver-builds/

Thanks!  Windows builds are already available in <https://build.mozilla.org/tryserver-builds/2008-01-23_05:24-hwaara@gmail.com-private_browsing_wip11/>.
(In reply to comment #103)
> (In reply to comment #101)
> > I gave the WIP 11 patch to the try-server, so a private browsing-build for all
> > three platforms should appear in about 1 hour and 30 minutes here:
> > https://build.mozilla.org/tryserver-builds/
> 
> Thanks!  Windows builds are already available in
> <https://build.mozilla.org/tryserver-builds/2008-01-23_05:24-hwaara@gmail.com-private_browsing_wip11/>.

Both Linux and OS X had build failures. See http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTry

(In reply to comment #104)
> Both Linux and OS X had build failures. See
> http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTry

Oops.  This should fix the problem.  Can you submit a new try server build?
Attachment #298699 - Attachment is obsolete: true
Oops, the previous patch accidentally contained some other code that I'm working on...  This one should really fix the problem :-)
Attachment #298729 - Attachment is obsolete: true
I submitted it - should show up in the same place when it's all set.
(In reply to comment #107)
> I submitted it - should show up in the same place when it's all set.

Thanks!  Windows and Mac builds have finished: <https://build.mozilla.org/tryserver-builds/2008-01-23_10:16-swilsher@mozilla.com-Bug_248970/>.  Linux is still building.  I'll monitor the tinderbox to see if it succeeds.

And, I'm preparing the final version of the patch which is ready for review.  It should be ready in a few minutes.  This version of the patch corrects the nsCertOverrideService implementation, and now it actually works!  It would be great if someone could submit another try build once I submit that patch.
Attached patch Patch (v1) (obsolete) — Splinter Review
OK, I think this is ready for review.

Changes in this version: the nsCertOverrideService implementation has been corrected as follows:

* I used NS_WITH_PROXIED_SERVICE() to proxy the calls to the nsIObserverService to the main thread, so that they won't fail, and notifications are received.

* I implemented nsISupportsWeakReference so that calls to nsIObserverService::AddObserver() with the last param set to true succeed.

(Note that this effectively fixes bug 413627 as well.)

-----

Note to reviewers:

1. The changes to browser-menubar.inc should not be reviewed.  Once this is ready to land, I'll remove the changes to that file.  Those are only meant to ease your life in testing the patch.

2. If you feel you won't be able to review this soon, please let me know to ask someone else for review.

-----

Asking Mike Connor to review the following parts:
toolkit/components/places/src/nsNavHistory.{h,cpp}
toolkit/components/contentprefs/src/nsContentPrefService.js
browser/components/privatebrowsing

Asking Benjamin Smedberg for sr on the following parts:
netwerk/{cache,cookie}/*
extensions/cookie/*
security/manager/ssl/*
xpcom/base/*

Asking Mike Beltzner for ui-r based on the descriptions in the previous comments.
Attachment #298732 - Attachment is obsolete: true
Attachment #298757 - Flags: ui-review?(beltzner)
Attachment #298757 - Flags: superreview?(benjamin)
Attachment #298757 - Flags: review?(mconnor)
Blocks: 413627
No longer depends on: 413627
CCing the relevant people from bug 413627...
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

Asking Benjamin Smedberg for review on the following parts:

xpcom/base/nsIConsoleService.idl
xpcom/base/nsConsoleService.{h,cpp}
Attachment #298757 - Flags: review?(benjamin)
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

Asking Simon Bünzli for review on the following parts:

browser/components/sessionstore/src/nsSessionStore.js
Attachment #298757 - Flags: review?(zeniko)
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

Asking Christian Biesinger for review on the following parts:

netwerk/cache/src/nsCacheService.{h,cpp}
Attachment #298757 - Flags: review?(cbiesinger)
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

Asking Dan Witte for review on the following parts:

netwerk/cookie/src/nsCookieService.{h,cpp}
extensions/cookie/nsPermissionManager.{h,cpp}
Attachment #298757 - Flags: review?(dwitte)
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

Asking Bob Relyea for review on the following parts:

security/manager/ssl/src/nsCertOverrideService.{h,cpp}
Attachment #298757 - Flags: review?(rrelyea)
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

Asking Asaf Romano for review on the following parts:

toolkit/components/satchel/src/nsStorageFormHistory.{h,cpp}
Attachment #298757 - Flags: review?(mano)
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

Asking Shawn Wilsher for review on the following parts:

toolkit/components/downloads/src/nsDownloadManager.{h,cpp}
Attachment #298757 - Flags: review?(sdwilsh)
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

Asking Justin Dolske for review on the following parts:

toolkit/components/passwordmgr/src/nsLoginManagerPrompter.js

(Oh, boy, I really wish Bugzilla could handle multiple review requests...)
Attachment #298757 - Flags: review?(dolske)
Targetting beta3...
Target Milestone: Future → Firefox 3 M11
(In reply to comment #118)
> (Oh, boy, I really wish Bugzilla could handle multiple review requests...)

It can. Just comma-separate the addresses. :)
(In reply to comment #120)
> It can. Just comma-separate the addresses. :)

And if only I knew it 11 comments ago...  ;-)

Thanks for the tip, Reed!
I must say, the approach implemented in the patch concerns me. You are modifying code in a lot of modules and try to prevent it from saving.

I think, using this approach, we will have to play "catch up" forever.
Whenever someone implements storage for some new data, you will have to enhance that storage logic to consider the proposed new mode.

I'm also concerned that it complicates the code.


I would like to bring up 2 alternative ideas.

a) Integrate your "private surfing mode" with profile manager. In other words, if the user chooses to surf privately, create a new profile. Mark that profile as such. During app shutdown, erase the profile. At worst the app crashes, so you could auto-cleanup the profile on next startup.

But a) does not work if you would like to prevent that data reaches the disk. In that case one could think of the following:


b) Maybe we could use an in-memory-virtual-filesystem? Implement file I/O that simulates a profile directory and files that live in RAM? Start a profile with a directory stored on that filesystem. This mechanism would guarantee we will never forget to adjust code that potentially stores sensitive data on disk (and keeps the code simple).
(In reply to comment #122)
> I must say, the approach implemented in the patch concerns me. You are
> modifying code in a lot of modules and try to prevent it from saving.
> 
> I think, using this approach, we will have to play "catch up" forever.
> Whenever someone implements storage for some new data, you will have to enhance
> that storage logic to consider the proposed new mode.

That's correct, but things like bug 413659 could help remedy the problem (see comment 94).

> I'm also concerned that it complicates the code.
> 
> 
> I would like to bring up 2 alternative ideas.
> 
> a) Integrate your "private surfing mode" with profile manager. In other words,
> if the user chooses to surf privately, create a new profile. Mark that profile
> as such. During app shutdown, erase the profile. At worst the app crashes, so
> you could auto-cleanup the profile on next startup.
> 
> But a) does not work if you would like to prevent that data reaches the disk.
> In that case one could think of the following:

Yes, (a) won't work for private browsing.

> b) Maybe we could use an in-memory-virtual-filesystem? Implement file I/O that
> simulates a profile directory and files that live in RAM? Start a profile with
> a directory stored on that filesystem. This mechanism would guarantee we will
> never forget to adjust code that potentially stores sensitive data on disk (and
> keeps the code simple).

That's a good idea as well, but I'm wondering how much could should be touched to implement that one.  I'm not quite familiar with Mozilla's I/O abstraction layers, but if this needs to be done at a level similar to nsILocalFile, then I'm afraid that the same amount of changes need to be made to these modules, to switch to an alternative file implementation.  Others may be able to comment on this more precisely.
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

Switching review of security/manager/ssl/src/nsCertificateOverrideService.{h,cpp} since he's the original author of the code that is modified.

Kai feel free to kick it back to me if you can't get to the review.

Ehsan, You man also want to open NSS Read/Only. That will prevent manual changes and new Root cert imports (or even private key/user cert importing) from happenning in private sessions.

bob
Attachment #298757 - Flags: review?(rrelyea) → review?(kengert)
(In reply to comment #119)
> Targetting beta3...

Just to set expectations properly, I don't think that we should count on this feature making it into Firefox 3. I say this because ..:

 - this bug isn't marked blocking- nor wanted-firefox3+
 - QA hasn't been planning on testing this mode
 - all of the reviewers will be focusing on blocking bugs first
 - some of Kai's comments make me think that there might be more thinking to be done here

This is great stuff, and I think a fantastic and huge step towards a much desired ability to do true private browsing. I'm looking forward to playing with the tryserver builds, and am truly appreciative of all the work put into this, and by no means am suggesting we abandon things. Just that we make sure to not get disappointed if it misses.

(That's why I was asking about the ability to get this done as an add-on, though I understand that might be difficult due to the nature of the changes - tryserver builds help!)
(In reply to comment #124)
> (From update of attachment 298757 [details] [diff] [review])
> Switching review of
> security/manager/ssl/src/nsCertificateOverrideService.{h,cpp} since he's the
> original author of the code that is modified.
> 
> Kai feel free to kick it back to me if you can't get to the review.
> 
> Ehsan, You man also want to open NSS Read/Only. That will prevent manual
> changes and new Root cert imports (or even private key/user cert importing)
> from happenning in private sessions.

Hmmm, I think private browsing is about not collecting data automatically about user's private session, but some things should be allowed anyways, such as bookmarking, saving a web page, etc.  Therefore I doubt that we should go as far as opening NSS read-only...

(In reply to comment #122)
> b) Maybe we could use an in-memory-virtual-filesystem? Implement file I/O that
> simulates a profile directory and files that live in RAM? Start a profile with
> a directory stored on that filesystem. This mechanism would guarantee we will
> never forget to adjust code that potentially stores sensitive data on disk (and
> keeps the code simple).

Another thing which needs clarification here is that whether we want to give user a new profile, therefore depriving them of everything useful in their real profiles?  I'd expect users to want access to their bookmarks, or their custom search engines for example n private browsing mode, to get started browsing...
I resubmitted the latest patch to the tryserver, and am playing with the current one for now ...
(In reply to comment #126)
> Hmmm, I think private browsing is about not collecting data automatically about
> user's private session, but some things should be allowed anyways, such as
> bookmarking, saving a web page, etc.  Therefore I doubt that we should go as
> far as opening NSS read-only...

I think that's right. Saving web pages, adding bookmarks, and other explicit user actions asking to save data should be honoured. It's the automatic stuff we do to make the browser more usable: retaining history, saving form history, accepting/sending cookies ... that's the sort of stuff we want to restrict.

> Another thing which needs clarification here is that whether we want to give
> user a new profile, therefore depriving them of everything useful in their real
> profiles?  I'd expect users to want access to their bookmarks, or their custom
> search engines for example n private browsing mode, to get started browsing...

That's a really easy way to get a bare-bones private browsing mode installed, but it isn't very functional.

I'd rather see Kai's concerns addressed while maintaining this approach, fwiw.
(In reply to comment #122)
> I think, using this approach, we will have to play "catch up" forever.

Not only us. What about an extension which saves urls to prefs (e.g. NoScript) or to their own data file (e.g. Adblock Plus and Tab Mix Plus)? Will somebody from MoCo go through AMO and make sure that that's taken care of for all extensions - or will private browsing only be really supported for extension-less profiles after all (just to be sure)?
(In reply to comment #125)
> (In reply to comment #119)
> > Targetting beta3...

Thanks for the supportive comments, Mark!

> Just to set expectations properly, I don't think that we should count on this
> feature making it into Firefox 3. I say this because ..:
> 
>  - this bug isn't marked blocking- nor wanted-firefox3+
>  - QA hasn't been planning on testing this mode
>  - all of the reviewers will be focusing on blocking bugs first
>  - some of Kai's comments make me think that there might be more thinking to be
> done here
> 
> This is great stuff, and I think a fantastic and huge step towards a much
> desired ability to do true private browsing. I'm looking forward to playing
> with the tryserver builds, and am truly appreciative of all the work put into
> this, and by no means am suggesting we abandon things. Just that we make sure
> to not get disappointed if it misses.

So, if I can get all of the reviews in time, would this be possible to land this for Firefox 3?  (I'm most worried about the "QA hasn't been planning on testing this mode" part.)

Also, I guess this won't be something which could be expected to land in a 3.0.x release, right?  All this haste is because I know many users which would appreciate a lot if their browser provided private browsing mode capability.  I personally use a script to launch Firefox with a new profile (and the -no-remote command line param) and delete it after the browser exits, but I'm sick and tired of it!  :-)

> (That's why I was asking about the ability to get this done as an add-on,
> though I understand that might be difficult due to the nature of the changes -
> tryserver builds help!)

If there's some way of increasing the users testing the tryserver builds, please let me know.  I'm running one myself (of course!).

Anyway, would it be OK to dance the Target Milestone field to match the next milestone until the Firefox 3 release and then move it to Future if it misses?  ;-)
(In reply to comment #127)
> I resubmitted the latest patch to the tryserver, and am playing with the
> current one for now ...

Should it be available at <https://build.mozilla.org/tryserver-builds/>?  (It's not currently.)
(In reply to comment #129)
> (In reply to comment #122)
> > I think, using this approach, we will have to play "catch up" forever.
> 
> Not only us. What about an extension which saves urls to prefs (e.g. NoScript)
> or to their own data file (e.g. Adblock Plus and Tab Mix Plus)? Will somebody
> from MoCo go through AMO and make sure that that's taken care of for all
> extensions - or will private browsing only be really supported for
> extension-less profiles after all (just to be sure)?

Naive (and non-practical) answer: the extensions can be modified to use the browser:private-browsing notification.

More reasonable: warn the user about the privacy concerns of such extensions.  Should this warning be in the UI, or in the docs, or both?

Imaginative answer: would it be possible to make some changes to the core Mozilla I/O code, so that when the private browsing mode is turned on, the I/O subsystem (part of it which handles in-profile data files) creates a new access path to the profile data: a copy-on-write (COW) scheme, which copies the modified data to memory, and reads such data from memory later on, until the private mode is switched off, which would cause the in-memory copies of data to be discarded, and the I/O handles restored to make further I/O happen on disk like usual?

Open questions:
1) Is this possible at all, from an architectural point of view?  (Are there any high-level docs available on the I/O architecture in Mozilla, designed for someone who's not a guru in that field, i.e., me?)
2) What to do with APIs which can't make the above requirement?  (What happens when a file grows in private browsing mode, and some code seeks into the newly appended portion, for example?)
3) Would it be possible to implement this from a logical point of view?  (How would code react to gets its storage "swapped" in and out from underneath it?)
4) How much of the code which accesses the filesystem uses Mozilla's I/O facilities?  (For example, would this cover code which stores data in sqlite)?
(In reply to comment #131)
> (In reply to comment #127)
> > I resubmitted the latest patch to the tryserver, and am playing with the
> > current one for now ...
> 
> Should it be available at <https://build.mozilla.org/tryserver-builds/>?  (It's
> not currently.)

Didn't compile due to a bad checkin from Places, trying again now, should appear with the bug number and "Patch v1" in the build name.
Comment on attachment 298757 [details] [diff] [review]
Patch (v1)

>+  mInPrivateBrowsing = PR_FALSE;
That should be in the constructor (which you'll have to add).

You also don't need to remove the observer.
Attachment #298757 - Flags: review?(sdwilsh) → review+
Target Milestone: Firefox 3 beta3 → Firefox 3 beta4
Attached patch Patch (v1.1) (obsolete) — Splinter Review
Unbitrotted patch.  Addressed the issues in comment 135.  I also removed the RemoveObserver calls for places where observers were added using weak references.
Attachment #298757 - Attachment is obsolete: true
Attachment #301098 - Flags: superreview?(benjamin
Attachment #301098 - Flags: review?(zeniko)
Attachment #301098 - Flags: review?(mconnor)
Attachment #301098 - Flags: review?(mano)
Attachment #301098 - Flags: review?(kengert)
Attachment #301098 - Flags: review?(dwitte)
Attachment #301098 - Flags: review?(dolske)
Attachment #301098 - Flags: review?(cbiesinger)
Attachment #301098 - Flags: review?(benjamin)
Attachment #298757 - Flags: ui-review?(beltzner)
Attachment #298757 - Flags: superreview?(benjamin)
Attachment #298757 - Flags: review?(zeniko)
Attachment #298757 - Flags: review?(mconnor)
Attachment #298757 - Flags: review?(kengert)
Attachment #298757 - Flags: review?(dwitte)
Attachment #298757 - Flags: review?(cbiesinger)
Attachment #298757 - Flags: review?(benjamin)
Attachment #298757 - Flags: review?(
Comment on attachment 301098 [details] [diff] [review]
Patch (v1.1)

In case you really want to go through with the issue this way:

>Index: browser/components/sessionstore/src/nsSessionStore.js
>===================================================================
>@@ -184,16 +189,18 @@ SessionStoreService.prototype = {
>+
>+    this._inPrivateBrowsing = false;

This isn't needed as _inPrivateBrowsing is already initialized to false and we only get through here once (and even _if_ we got through here again, we'd surely not want to bail out of private browsing mode).

>-    if (this._sessionFile.exists()) {
>+    if (this._sessionFile && this._sessionFile.exists()) {

What's this change for? _sessionFile and _sessionFileBackup should always be initialized at this point.

>+    var backup = [];

You're using this as a hash, not an array, so please make it a {}. (In fact, using |for (... in ...)| is highly discouraged for arrays while it's OK for iterating over the keys of a hash.)

>+    for (ix in this._windows)
>+      backup[ix] = eval(this._windows[ix]._closedTabs.toSource());

Nits: ix is never declared. And please add a comment as to why you're doing |eval(....toSource())|.

>+    for (ix in this._closedTabsBackup) {
>+      if (this._windows[ix]) {

Nits: ix isn't declared. And you can drop the braces for the if clause (as in one or two further places).

r+=me for the SessionStore changes with these issues fixed.
Attachment #301098 - Flags: review?(zeniko) → review+
i'm in favor of having this be a notification rather than a pref, but some services are lazily inited (nsCookieService, nsPermissionManager, most likely others). for instance, |firefox about:blank| won't init the cookieservice, and if the user enters private browsing they'll miss that notification. any ideas? having the cookieservice call up |nsIPrivateBrowsingService::GetPrivateBrowsing()| on init kinda sucks :/
Why does that suck? I don't think we're going to have any choice but have a global mode flag *somewhere* which indicates whether we are "right now" in private browsing mode. I'm happy to hang that on nsIXULRuntime or a new interface, whichever is easier. Then we have global observer topics for when the private-browsing-mode changes.

This bug is huge, by the way. Could we document the current set of design decisions in the wiki page? Especially the invariants we're trying to preserve (no website data to disk during private-browsing), and how that actually affects cookie/cache/etc.
(In reply to comment #139)
> Why does that suck?

it's another interface dependency added to all these services - not especially bad, given that we're adding all this PB code anyway, it just means nsIPB needs to live early in the build process. (i.e. before netwerk.) if not, something more intuitive than nsIXULRuntime might be nice, <shrug>...

and agreed, some concrete documentation to refer back to here during discussion would be awesome.
Maybe this should be a separate bug/enhancement request, but it would be nice to be able to clear a particular item from one's "saved form" data without having to clear the entire history, for example when one inadvertently types a password into the login-name field.
Ah, thanks.  I had been trying BACKSPACE and nothing happened.
instead of using DELETE, that is.
Target Milestone: Firefox 3 beta4 → ---
Comment on attachment 301098 [details] [diff] [review]
Patch (v1.1)

r- because this doesn't have tests and doesn't have a design doc
Attachment #301098 - Flags: superreview?(benjamin)
Attachment #301098 - Flags: superreview-
Attachment #301098 - Flags: review?(benjamin)
Attachment #301098 - Flags: review-
Should this be blocked by Bug 290456 ?
(In reply to comment #145)
> Should this be blocked by Bug 290456 ?

I think it would be desirable to incorporate any solution developed for bug 290456 in Private Browsing, but I don't think it should block Private Browsing, especially because we don't currently handle clearing that sort of data right now...
Hello, I'm the developer of the Torbutton Firefox extension (https://www.torproject.org/torbutton), and I've put a great deal of work and research (about 16 months now) into developing a 'private mode' via extensionland for Tor users. I approached the problem from the point of view that any amount of data that the browser leaks on to the disk or the network that is either uniquely identifying, can be used to build a random identifier, or indicative of their history is a vulnerability (See https://www.torproject.org/torbutton/design/#adversary and https://www.torproject.org/torbutton/design/#requirements).

While this was a different approach that seems to have been taken from this bug (which seems to have been attacked primarily from a usability perspective), it seems to have led us to similar conclusions.

I've documented everything I have learned and implemented in the Torbutton design document here: https://www.torproject.org/torbutton/design/

A couple of comments I can give from experience that weren't covered in your above discussion, or that I'd just like to re-emphasize:

0. Pages loaded in non-private mode can do all sorts of automatic network activity, set timers, attempt to re-authenticate, send unique identifiers stored in 'DOM Storage', etc etc during private mode.

1. Content window Javascript can access session history to navigate back and forward for the user to cause the local network (in our case the exit node) to learn a bit about their most recently accessed private or non-private session history.

2. Formfill and password saving can write private data to disk, and read it back where content window Javascript can inspect the values.

3. Livemarks refreshes currently cannot be disabled, and will still happen in the background during private mode (Bug 436250), exposing potentially private data to the local network (such as wikipedia pages you are an editor of, etc)

4. Plugins have their own cookies that can still be transmitted in private mode (see http://epic.org/privacy/cookies/flash.html).

5. Users hate it when a private mode does anything to permanently alter any of their non-private data. The only way to do this and have it ever be used is to isolate the non-private data so that it can't be touched during private mode, but is restored upon entrance back into non-private mode.

6. Custom SSL certificates are still available and can be probed for.

7. (Already covered briefly, would like to re-emphasize): History disclosure is a big big problem (see http://gemal.dk/browserspy/css.html and http://ha.ckers.org/weird/CSS-history.cgi http://www.mikeonads.com/2008/07/13/using-your-browser-url-history-estimate-gender/), 
but it must not be solved by damaging the user's current stored history, or even 
making it inaccessible.

8. Private mode should also be a pref in addition to an observer event, so that components and windows loaded after the transition can still determine the current mode.


I've also cataloged a list of Firefox bugs that make much of the above difficult under the current version of Firefox: https://www.torproject.org/torbutton/design/#FirefoxBugs

Flags: in-litmus?
A while back Beltzner told me that the drivers feel that the risk in the approach I've took in my patches on this bug is too high and that they're more willing to take a Places-based solution.  Are there any updates on this?  Or should I just pick up my slack here and work on it more?
>Are there any updates

I'll try to get an answer to you soon about this
Ping?
According to <https://wiki.mozilla.org/Firefox3.1/Features>, this seems to have been dropped from 3.1.  Anyway, it would be nice to know what the plans here are for this to get into the next release.
Too bad- the only new feature I was really looking for :(
A Slashdot article today says Microsoft is applying for some patents on private browsing.
(In reply to comment #153)
> A Slashdot article today says Microsoft is applying for some patents on private
> browsing.

Seems like there was some confusion, it appears it's only about a trademark for product names.
Adding Wan-Teh to cc list, as a functionality that uses a memory-only filesystem would probably involve new functionality at the NSPR level.
>Ping?

Sorry about the lag, I thought mconnor had been in touch with you.  Recent development with Chrome will likely make finally getting private browsing mode shipped a priority for 3.1, but I think we are now targeting a more lightweight implementation.

What we need to figure out is if you should continue your implementation for use in a later release.

Here are some details mconnor sent me in email a few days ago about a possible strategy for 3.1:

>Main goals:
>
>Ensure that users can't be tracked when doing "private" things.  There
>should be a clear line drawn between your "public" and "private"
>browsing sessions.  It is acceptable to let things touch magnetic
>storage, as long as the cleanup mechanism is robust enough to clean
>up.
>
>It is also acceptable to retain data that users explicitly save
>(per-site permissions via prefs, bookmarks, etc)
>
>Non-goal for 3.1: Separate process sharing (some) data.  When we get
>process-per-tab we can make it more IE-like, but doing this also means
>that we have to have something like their "hey, you're in private
>browsing mode" banner on the URL bar for all the world to see.  Which,
>to me, is fail.
>
>
>Cookies:
>
>On entry:
>
>Write cookies to disk, drop the in-memory hashtable.
>
>During:
>
>All cookies are treated as session cookies.
>
>Exit:
>
>Drop the hashtable, reload from disk.
>
>
>History:
>
>On entry:
>
>Record timestamp of the last visit recorded.
>
>During:
>
>IsVisited always returns false (no link coloring spying)
>AddVisit silently fails.
>
>Exit:
>
>Ensure any visits recorded after the timestamp are purged (shouldn't
>be needed, but might be useful as a sanity check).
>
>
>Site Permissions:
>
>Page Info tab is disabled.
>Will not prevent users from explicitly adding exceptions via prefs.
>
>Passwords:
>
>Do not prompt to save passwords.
>Passwords will not autofill, but will be available for autocomplete.
>
>Other:
>
>Autocomplete will be available, but will not remember data entered.
>DOMStorage will not allow reading or writing of data (need JST/Enn
>feeedback on how to do this cleanly)
>All authenticated sessions will be logged out entering and leaving
>private mode.
>Downloads will be removed from dlmgr on completion.
>
>
>Optional: Save session and close all browser windows, and restore after
>exiting private mode?  Seems reasonable enough, especially if we can
>add the session store override to save SSL form data as a one-off...
I like the direction this is headed, especially if any of it shows up in 3.1 but I would like to suggest that IsVisited always returning false be rethought.

I find link coloring to be quite useful, particularly when going one by one through a list of links (e.g., on Google or a shopping site).

If it a matter of trying to figure out how to keep this off the disk and would require major re-work of that code, I can understand pushing it out until a later version but, to me, this is a major disadvantage of private mode: it's my biggest complaint about Stealther.
How about a second set of files for private browsing that automatically get scrubbed.  Alternatively, an indicator (or column) in the sqllite DB that marks something as from the private session.
(In reply to comment #158)
> Alternatively, an indicator (or column) in the sqllite DB that marks
> something as from the private session.

This sounds me like session history, just like session cookies.
I think Distrust has the best implementation of private browsing. It has an icon at the bottom which you click to enter a session. Then when you have finished, you unclick it and your previous tabs are restored and no history is saved.

For cookies, they are treated as session cookies and deleted at shutdown.

https://addons.mozilla.org/en-US/firefox/addon/1559

Contact site for the maker

http://www.gness.com/distrust/
Attached patch Patch (v2.0) (obsolete) — Splinter Review
This patch implements the Private Browsing mode as described in comment 156.  All parts of the requirements are met here, except the DOM storage, which I'd be happy to have a feedback from JST or Enn on how best to tackle this in that module as neatly as possible.

Some parts of the code have not changed from the previous WIP patches, some other parts have been removed now that our requirements are a bit lighter, and some rewriting has also been performed.  It would be great if mconnor and other could test this both functionality and code-wise.  Also, I'd like to have beltzner's input on this as well, both on how the private browsing mode with the approach in comment 156 feels, and also on our chances to have this for 3.1.

It would also be great if someone with appropriate access could submit this patch to the try server for the users to try out actual builds.

Last, but not least, like with the previous patches, the current menu item is a mortal and mere hack which will be removed from the final version of this patch.  The UI should be decided and implemented in bug 411929.
Attachment #301098 - Attachment is obsolete: true
Attachment #301098 - Flags: ui-review?(beltzner)
Attachment #301098 - Flags: review?(mconnor)
Attachment #301098 - Flags: review?(mano)
Attachment #301098 - Flags: review?(kaie)
Attachment #301098 - Flags: review?(dwitte)
Attachment #301098 - Flags: review?(dolske)
Attachment #301098 - Flags: review?(cbiesinger)
Attachment #337327 - Flags: ui-review?(beltzner)
(In reply to comment #161)
> Created an attachment (id=337327) [details]
> Patch (v2.0)
> 
> This patch implements the Private Browsing mode as described in comment 156. 
> All parts of the requirements are met here, except the DOM storage, which I'd
> be happy to have a feedback from JST or Enn on how best to tackle this in that
> module as neatly as possible.

It already supports session storage via permissions like cookies do. You should be able to either set that or modify nsDOMStorage::CanUseStorage as needed.

Dave Camp is the current owner of the code.
(In reply to comment #162)
> It already supports session storage via permissions like cookies do. You should
> be able to either set that or modify nsDOMStorage::CanUseStorage as needed.
> 
> Dave Camp is the current owner of the code.

A quick look at CanUseStorage() and it seems to be only used in one place in nsDOMStorage implementation.  Is this enough to make this function return false during the private browsing session so that websites can't access or store data in this mode?
Was cache handling dropped from the design? It seems like another thing we should look after in priv browsing.
(In reply to comment #164)
> Was cache handling dropped from the design? It seems like another thing we
> should look after in priv browsing.

Yes, I think so too.  I had an implementation of cache handling in the previous patch which I deleted out.  I'm not sure about the rationale for dropping cache handling from the design though.  Mconnor, can you elaborate please?
I prepared builds for Windows and Linux with the latest patch:

<http://ehsanakhgari.org/blog/2008-09-08/private-browsing-builds-ready>

Please take some time to test and evaluate it.  Also, if anyone owns a Mac and is willing to create a Mac build as well, that would be great!
(In reply to comment #161)
> Created an attachment (id=337327) [details]
> Patch (v2.0)
...
> It would also be great if someone with appropriate access could submit this
> patch to the try server for the users to try out actual builds.

Just submitted to try server.  Assuming it builds cleanly, it should show up here in an hour or two:

https://build.mozilla.org/tryserver-builds/?C=M;O=D
(In reply to comment #167)
> Just submitted to try server.  Assuming it builds cleanly, it should show up
> here in an hour or two:
> 
> https://build.mozilla.org/tryserver-builds/?C=M;O=D

<http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1220880196.1220880386.10390.gz>

Seems like a wrong -p option to patch.  patch should be invoked with -p1, I guess.  Identical results on Win32 and Mac.  Can you please re-submit the patch to try server?
I re-submitted the patch.
Assignee: ehsan.akhgari → nobody
Component: Menus → General
QA Contact: menus → general
Assignee: nobody → ehsan.akhgari
FWIW, spec I wrote is https://wiki.mozilla.org/User:Mconnor/PrivateBrowsing

If this patch is working to implement this spec, fantastic!  I'll take a look through tonight.  We can and will get this into 3.1 one way or another.
(In reply to comment #170)
> FWIW, spec I wrote is https://wiki.mozilla.org/User:Mconnor/PrivateBrowsing

Thanks for the link.  For reference of those who want to test this patch without building it themselves, here is the link to the try server builds:

<https://build.mozilla.org/tryserver-builds/2008-09-08_11:19-dgottwald@mozilla.com-1220897935/>

> If this patch is working to implement this spec, fantastic!  I'll take a look
> through tonight.

Yes, it does.  Originally I based my work on comment 156, but this spec is somewhat different to that (IINM only with regard to the Permissions section).  I can easily remove the pageinfo-specific parts from this patch, which should bring this patch close to the spec you wrote.  The only remaining part is DOM Storage handling.  Enn provided some feedback on it (comment 162), and I'm waiting for more feedback from Dave Camp (comment 163).

Also, I think we should add cache handling to the spec, because sensitive data about user's private session could be stored on disk as part of the cache, and accessing that data isn't that hard (doesn't even require manual traversal of cache data files, about:cache comes to rescue!).

Please let me know what you think!

> We can and will get this into 3.1 one way or another.

That's very good to hear.  I'm ready to work on this full-speed, because I guess it requires a good amount of time for QA after landing.
(In reply to comment #163)
> (In reply to comment #162)
> A quick look at CanUseStorage() and it seems to be only used in one place in
> nsDOMStorage implementation.  Is this enough to make this function return false
> during the private browsing session so that websites can't access or store data
> in this mode?

Yeah, that should work fine.  Most entry points call CaheStoragePermissions(), which calls (and doesn't actually cache, apparently) CanUseStorage().
Attached patch Patch (v2.1) (obsolete) — Splinter Review
Changes in this patch:

 * The page info stuff was removed, as per the spec.
 * The passwordmgr code was changed because my previous patch made it dependent on the browser code which is not acceptable.  I also removed the PrivateBrowsingListener.jsm code entirely (which was a stupid idea of mine to begin with -- it can all be done using the Private Browsing service itself!).
 * The DOM Storage related code is now implemented.

One thing that I'm not sure and I need Dave's feedback on is whether the DOM Storage manager gets initialized at app startup or is it possible to be initialized lazily?  If the latter is the case, I need to add some code to detect at initialization time whether the private browsing mode is on or off, which may cause me to revive a trick I made in one of the previous WIPs (for supporting nsISupportsPRBool by the Private Browsing service, because dom code can't be made dependent on browser in which the nsIPrivateBrowsingService interface is defined).  It would be trivial, so I'm ready to provide a new patch if Dave confirms that it's possible for the DOM Storage manager to be initialized *after* the user has already entered the private browsing mode.

Like before, feedback and try server builds are welcome! :-)

One final note: I still think that we need cache handling here...
Attachment #337327 - Attachment is obsolete: true
Attachment #337327 - Flags: ui-review?(beltzner)
It is important for extensions to be able to interact with the private browsing mode and capture its notifications.  I wrote a wiki page explaining the APIs for extensions and provided a number of code samples.  This can later be moved on to MDC.  You can see the page here: <https://wiki.mozilla.org/User:Ehsan/PrivateBrowsingForExtensions>

Obviously it's just a draft at this stage, and I'll keep on updating it as the API changes in this bug.
(In reply to comment #173)

>  * The passwordmgr code

A few comments:

* Didn't an earlier version of this feature fire a notification when private browsing mode was entered or left? Making the pwmgr poll the service during pageload is kind of sucky.

* It would be cleaner to have the pwmgr not try to prompt for saving/changing a login in the first place, instead of having the code in nsLoginMangerPrompter.js silently bail out when it's attempted.

* The spec says form logins should have the autocomplete attached, but not autofilled. For HTTP auth, should the popup dialog be prefilled? Otherwise such saved logins are inaccessible. The user can click cancel to not authenticate with the site, although I can see an argument for that being too easy to accidentally forget. :)
(In reply to comment #173)
> One thing that I'm not sure and I need Dave's feedback on is whether the DOM
> Storage manager gets initialized at app startup or is it possible to be
> initialized lazily?  If the latter is the case, I need to add some code to
> detect at initialization time whether the private browsing mode is on or off,
> which may cause me to revive a trick I made in one of the previous WIPs (for
> supporting nsISupportsPRBool by the Private Browsing service, because dom code
> can't be made dependent on browser in which the nsIPrivateBrowsingService
> interface is defined).  It would be trivial, so I'm ready to provide a new
> patch if Dave confirms that it's possible for the DOM Storage manager to be
> initialized *after* the user has already entered the private browsing mode.

The storage manager is initialized at startup.

But if pieces below the browser are using this feature, wouldn't it make sense to put nsIPrivateBrowsingService.idl somewhere lower in the stack?  It doesn't seem like it's any cleaner to have everything built before browser/ either be initialized at startup or use a different interface to access the service as anything after browser/.
Comment on attachment 337732 [details] [diff] [review]
Patch (v2.1)

So, the download manager may or may not be initialized at startup, so you can't depend on receiving the notification.

Secondly, this really needs some tests.  Toolkit has a requirement for any new feature to land with tests, and I think browser may be the same way.

You won't get my review for the download manager bits without a test to show that it works as expected.
(In reply to comment #177)
> You won't get my review for the download manager bits without a test to show
> that it works as expected.

... but we really hope you will supply those tests, because the work so far is great, Ehsan.  Really - thank you for continuing to plug away at this.
(In reply to comment #177)
> So, the download manager may or may not be initialized at startup, so you can't
> depend on receiving the notification.

The typical pattern would be for a component to get the current state when it starts up, at the same time it registers an observer.
(In reply to comment #175)
> (In reply to comment #173)
> 
> >  * The passwordmgr code
> 
> A few comments:

Thanks for the notes! :-)

> * Didn't an earlier version of this feature fire a notification when private
> browsing mode was entered or left? Making the pwmgr poll the service during
> pageload is kind of sucky.

Yeah, it still does.  I'll change this in the next revision of the patch.

> * It would be cleaner to have the pwmgr not try to prompt for saving/changing a
> login in the first place, instead of having the code in
> nsLoginMangerPrompter.js silently bail out when it's attempted.

Agreed.  I'll try to move as much of the code as possible to nsLoginManager in
the next revision.

> * The spec says form logins should have the autocomplete attached, but not
> autofilled. For HTTP auth, should the popup dialog be prefilled? Otherwise such
> saved logins are inaccessible. The user can click cancel to not authenticate
> with the site, although I can see an argument for that being too easy to
> accidentally forget. :)

Well, that's really something for the spec to address.  Currently the patch
doesn't autofill form fields, and leaves HTTP auth dialogs blank.  I can easily
change this so that they get filled, provided that mconnor is in favor of this
change.
(In reply to comment #176)
> But if pieces below the browser are using this feature, wouldn't it make sense
> to put nsIPrivateBrowsingService.idl somewhere lower in the stack?  It doesn't
> seem like it's any cleaner to have everything built before browser/ either be
> initialized at startup or use a different interface to access the service as
> anything after browser/.

I also think browser/ is not a good place to put nsIPrivateBrowsingService.idl, but I'm not sure where to put it.  Can you suggest some place?  Ideally we need some place where we can make all of the modules dependent on it, so that they can all access nsIPrivateBrowsingService, but I'm not sure if such a place exists in the tree...
(In reply to comment #177)
> So, the download manager may or may not be initialized at startup, so you can't
> depend on receiving the notification.

Good point to mention.  Based on comment 179, I think I'm going to change the logic for each module to get the current status on startup (no matter if it's initialized at startup or not), and I'll make sure to handle download manager this way as well.

> Secondly, this really needs some tests.  Toolkit has a requirement for any new
> feature to land with tests, and I think browser may be the same way.
> 
> You won't get my review for the download manager bits without a test to show
> that it works as expected.

Sure.  The next thing I'll be looking into is getting started to write some tests for this code.  I'll try to make this in-litmus- and in-testsuite+.  :-)

(In reply to comment #178)
> ... but we really hope you will supply those tests, because the work so far is
> great, Ehsan.  Really - thank you for continuing to plug away at this.

Thanks!  :-)

(In reply to comment #179)
> The typical pattern would be for a component to get the current state when it
> starts up, at the same time it registers an observer.

I see.  I think before jumping at this, I'll need to find some place else to put nsIPrivateBrowsingService.idl in, so that all the modules can query the service on startup.  I need help finding the appropriate place, though.

If such a place cannot be found, I'm thinking of defining another notification, named something like "private-browsing-query" which passes a nsISupportsPRBool as the subject (like "quit-application-requested"), and make the private browsing service respond to this notification by passing the current status in aSubject.

Any hints on whether I'm barking at the right tree or not?
Flags: in-testsuite?
Ehsan, if it'd help I can write up a test plan for what we need to test.  If you want to work together on this, just find me on IRC.

(In reply to comment #157)
> I like the direction this is headed, especially if any of it shows up in 3.1
> but I would like to suggest that IsVisited always returning false be rethought.
> 
> I find link coloring to be quite useful, particularly when going one by one
> through a list of links (e.g., on Google or a shopping site).

Given that this can be (ab)used by sites, it doesn't make sense to expose your non-private history to them, and since we're not recording visits in private mode, there's nothing reasonable we can return.
I gave a try with the build linked in comment 171. I enabled private browsing, visited a given website, disabled it and grepped the profile for occurrences of that website URL. I could find references in the files:
profile/Cache/<a bunch of files>
profile/cookies.sqlite
profile/places.sqlite

(and in session_restore.js before I quit the browser). The cache and session restore are not handled yet, but shouldn't cookies and places not contain traces?
(In reply to comment #184)
> The cache and session
> restore are not handled yet, but shouldn't cookies and places not contain
> traces?

Thanks for testing this, Sylvian.  Yes, you're right, cookies and places are supposed not to leave a trace.  Can you open the sqlite DBs and check in which table(s) the mentioned URL appears please?
sure, I accessed slashdot with private browsing mode enabled:

sqlite3 cookies.sqlite .dump|grep slashdot
INSERT INTO "moz_cookies" VALUES(1221074391930408,'__utma','9273847.3921509624962351000.1221074392.1221074392.1221074392.1','.slashdot.org','/',1284146393,1221074393924407,0,0);
INSERT INTO "moz_cookies" VALUES(1221074391930799,'__utmb','9273847.1.10.1221074392','.slashdot.org','/',1221076193,1221074393926321,0,0);
INSERT INTO "moz_cookies" VALUES(1221074391932333,'__utmz','9273847.1221074392.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)','.slashdot.org','/',1236842391,1221074391932333,0,0);
INSERT INTO "moz_cookies" VALUES(1221074392683439,'__qca','1221074392-53816111-35021676','.slashdot.org','/',2147385600,1221074392683439,0,0);


sqlite3 places.sqlite .dump|grep slashdot
INSERT INTO "moz_favicons" VALUES(8,'http://slashdot.org/favicon.ico',X'000001000100101010000000000028010000160000002800000010000000200000000100040000000000C0000000000000000000000000000000000000000000000080800000FFFFFF00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000111000001110000011100001111100000111000111110000011100011111000000111000111000000011100000000000000111000000000000011100000000000000111000000000000011100000000000000111000000000000011100000000000000000000000000000000000000FFFF0000E7E70000C3C30000C3810000E1810000E1830000F0C70000F0FF0000F87F0000F87F0000FC3F0000FC3F0000FE1F0000FE3F0000FFFF0000FFFF0000','image/x-icon',1221160794701793);
INSERT INTO "moz_places" VALUES(45,'http://www.google.com/search?ie=UTF-8&oe=UTF-8&sourceid=navclient&gfns=1&q=slashdot','search','moc.elgoog.www.',1,0,0,NULL,100);
INSERT INTO "moz_places" VALUES(46,'http://slashdot.org/','Slashdot: News for nerds, stuff that matters','gro.todhsals.',1,0,0,8,100);
Thanks for the update, Sylvian.  The places part is half about the favicon service (which I didn't handle before, and should do so in the next revision).  The other half might be because of what I'm doing in AddVisit only makes sure nothing is recorded in moz_historyvisits, and moz_places would still be exposed to private data without further checks.

But I don't understand the cookies part at all.  Would you mind checking out to see if it's reproducible, both with the same profile and a new one, and post here a detailed STR here?  And also make sure you delete all this data before testing this again with the same profile (sorry for stating the obvious)?

Thanks!
(In reply to comment #187)
> But I don't understand the cookies part at all.

at first glance, the problem is you set an observer in the cookieservice, but it's inited lazily - not until first pageload. if Sylvain enabled PB before loading a page, cookieservice won't catch that notification. you'll need to do what dolske suggested in comment 179: check PB mode in Init() and also register an observer.
(In reply to comment #188)
> at first glance, the problem is you set an observer in the cookieservice, but
> it's inited lazily - not until first pageload. if Sylvain enabled PB before
> loading a page, cookieservice won't catch that notification. you'll need to do
> what dolske suggested in comment 179: check PB mode in Init() and also register
> an observer.

Thanks for the pointer, Dan!  Of course, you're right.  I'm going to work on this tomorrow.  If by that time someone doesn't come up with a solution to comment 181 (where to put nsIPrivateBrowsingService.idl), I'm going to take the "private-browsing-query" notification approach (comment 182)...
Here's the detailed steps:

Download and Extract the Firefox build somewhere
Create a new profile directory:
rm -rf /tmp/profile; mkdir /tmp/profile
Start it (adapt this for your OS)
./Minefield.app/Contents/MacOS/firefox -no-remote -profile /tmp/profile
Check Tools > Private Browsing
Open www.slashdot.org
Go back to Minefield Home page
Uncheck Tools > Private Browsing
Quit Firefox
Grep the profile:
grep -rli slashdot /tmp/profile
Expected: nothing
Actual: see comment 184


Minefield homepage (http://www.mozilla.org/projects/minefield/) does not set cookies, so Dan explanations apply in this situation.
(In reply to comment #190)
> ./Minefield.app/Contents/MacOS/firefox -no-remote -profile /tmp/profile

> Minefield homepage (http://www.mozilla.org/projects/minefield/) does not set
> cookies, so Dan explanations apply in this situation.

if that command loads the homepage on startup, that'll pull in the cookieservice (regardless of whether it sets anything... it needs to check if there are cookies to send, too). my explanation only applies if it loads about:blank.

something else may be amiss, throw in some debug statements and fiddle around?

(In reply to comment #189)
> comment 181 (where to put nsIPrivateBrowsingService.idl), I'm going to take 

it'll need to be somewhere above the netwerk tier, so toolkit's out, no?
I notice that neither the functional spec, the bug comments thus far, or the current patch seem to account for safebrowsing (anti-phishing/malware).

Presumably we still want to protect people in this mode from those attacks (indeed, they may be more likely to visit disreputable or compromised sites).  On the other hand, the pingback behaviour in the safebrowsing protocol means that when a reported malware/phishing site is hit, FF will go double-check, which means a hash request to google.  That is probably not behaviour that our users would expect, when in PB mode.

One reasonably straightforward technical solution there is to keep checking sites against the local DB, but not perform the pingbacks, while in PB mode.  Dave Camp (already cc'd) can comment on the technical feasibility there, but there is also a concern with that approach, namely that sites which have been removed from the list after the user enters PB mode will still be marked as dangerous.  We don't like to mark sites as dangerous that aren't actually.  Still, I'm not sure either of the other obvious solutions (disable malware/phishing protection entirely, or leave it entirely operational) are desirable.

I don't want to scope creep the bug - if we want to file a followup for this, we can, but I think we need an answer to it.  People using this mode are precisely the people who would be sensitive to automatic pingbacks.  (To be clear, these only occur for reported bad sites, not for every visit - a user might go their whole life without encountering a single one.  But then again, they may not. )
Blocking without a pingback would be feasible, but you'd end up with false positives in addition to stale blocks (not particularly often, but it could happen).

What might work is a separate error page for blocks that are unconfirmed due to private browsing mode.  It should be possible to add a "confirm this match" link that does the pingback manually.
Is there any hope to block content-prefs.sqlite from "leaking" pages visited in private browsing mode? Simply changing zoom level pushes the page to this database and I remember there were some difficulties removing it, even with "clear private data".
(In reply to comment #191)
> if that command loads the homepage on startup, that'll pull in the
> cookieservice (regardless of whether it sets anything... it needs to check if
> there are cookies to send, too). my explanation only applies if it loads
> about:blank.
> 
> something else may be amiss, throw in some debug statements and fiddle around?

I'll investigate this further.

> (In reply to comment #189)
> it'll need to be somewhere above the netwerk tier, so toolkit's out, no?

Yes.
(In reply to comment #192)
> People using this mode are
> precisely the people who would be sensitive to automatic pingbacks.

Hmmm, I was under the impression that safebrowsing only sent a hash digest of a URL to the provider when its (shorter) hash matches one of those downloaded from the provider periodically, right?  So there should not be anything revealing the URL that the user has visited in this process, right?

(In reply to comment #193)
> What might work is a separate error page for blocks that are unconfirmed due to
> private browsing mode.  It should be possible to add a "confirm this match"
> link that does the pingback manually.

If there are privacy concerns here, I think we can go with this solution, but I fail to see the privacy concerns anyway...  Anyway ultimately it's the call of someone more familiar with the code.
(In reply to comment #194)
> Is there any hope to block content-prefs.sqlite from "leaking" pages visited in
> private browsing mode? Simply changing zoom level pushes the page to this
> database and I remember there were some difficulties removing it, even with
> "clear private data".

An earlier revision of my patch handled content prefs as well.  Mconnor: was omitting the content prefs from the functional specs deliberate?

BTW, based on IRC talk with mconnor earlier, cache handling will be added to the patch, and a session store related change is also along the way.  The current plans are closing all of the windows/tabs when entering the private mode, and restoring them all after exiting the private mode, so that there is a clear separation between the private and non-private windows/tabs as far as the users are concerned.
(In reply to comment #196)
> (In reply to comment #192)
> > People using this mode are
> > precisely the people who would be sensitive to automatic pingbacks.
> 
> Hmmm, I was under the impression that safebrowsing only sent a hash digest of a
> URL to the provider when its (shorter) hash matches one of those downloaded
> from the provider periodically, right?  So there should not be anything
> revealing the URL that the user has visited in this process, right?

Your description of the double-check is correct:

 - User attempts to load a page whose (32-bit) hash matches one in the DB
 - We ask for the full length hash for that entry, to eliminate the possibility of collisions
 - At no point is the raw URL transmitted.

The concern arises because the list provider has all the "real" urls, so the usual difficulties of reversing a hash don't really apply.  If my client sends a double-check request, an unscrupulous list provider could figure out which of the entries in their list it corresponded to at which point they would know either that I had visited that site, or that I had visited a site which hash-collides with it.

That is a pretty minor leakage, particularly given the strength of the privacy policy google has in place around its collection of data, but it is non-negligible.  Dave's idea is probably a good one to consider as well, but yes, I think I will file a bug dependent on this one to track it, rather than risk derailing the progress here.
Bug 454792 opened to track the safebrowsing discussion.
Attached patch Patch (v2.2) (obsolete) — Splinter Review
What's new in this patch:

 * Added the "private-browsing-query" notification support to nsPrivateBrowsingService, and added correct initialization code to all the existing modules.  Order of initialization for modules is insignificant here: if a module initializes before the private browsing service, and sends the query notification, it will not be handled, so the module assumes that we're not in private browsing mode (correct assumption).  For all modules initialized after the private browsing service, the private browsing service will correctly respond to the query notification and the modules in question get initialized with the correct current private browsing status.
 * Handled Justin's first and second issues in comment 175.

Todo stuff for the next revision: add the cache (and content prefs service) handling, and improve the places handling of the private browsing mode (see comment 188).  Also unit tests are forthcoming, perhaps not in the next revision though (which I hope to get ready by tomorrow).
Attachment #337732 - Attachment is obsolete: true
Attached patch Patch (v2.3) (obsolete) — Splinter Review
What's new in this version of the patch:

 * Handled the cache service with this logic: on entering the private mode, disable the disk and offline caches.  During the private mode, only the memory cache remains active.  When leaving the private mode, clear the memory cache, and enable the disk and offline caches, if they should be enabled based on their respective prefs.
 * Handled the content prefs service, so that things such as changing the zoom level on a site doesn't cause trails of the visit be left in the profile.
 * Made the favicon service aware of the private mode, and disable saving of favicons (and potentially addition of moz_places entries as Sylvian had observed in comment 186).
 * Made sure that the test laid out in comment 190 passes successfully.

This patch is worth having a try server build for, so if someone can submit it, that would be great!

I'll get started with some unit tests in the next revision.
Attachment #338165 - Attachment is obsolete: true
(In reply to comment #201)
> Created an attachment (id=338299) [details]
> Patch (v2.3)

> This patch is worth having a try server build for, so if someone can submit it,
> that would be great!

https://build.mozilla.org/tryserver-builds/2008-09-12_06:19-dgottwald@mozilla.com-priv-20080912/

Note that Linux talos didn't like that build:
FAIL: Busted: tp
FAIL: browser frozen
I just want to confirm: This is not tab-independent, correct?  It applies to the whole browser.
While you are working on this project, I had an idea for an improvement or addition to it.

Add a feature that will allow users to clear all items instantly for the sessions they had browsed during the session of mozilla. I doubt you keep up a history of each cookie and history of what that individual process/tab has done during the whole time it has been open (would be a memory hog). Sometimes a user is browsing and might have forgotten to set mozilla for private browsing instead allowing users to clear the sessions, even if it only pertains to what is currently open and associated cookies/history.

any feedback against this is welcome, just wanted to get it out in the open. may not be the right forum either.
I just took a quick look over it and it looks awesome. Great work!

One concern: From my quick look it didn't seem to be touching nsSessionStore.js. Are closed tabs remembered using your current code? (Again, quick look and I haven't tried it) I ask because I'm implementing "undo closed window" which is analogous to the "undo closed tabs" feature, and I'm trying to get a feel for how this might effect the time line for that, and vice versa.
(In reply to comment #202)
> https://build.mozilla.org/tryserver-builds/2008-09-12_06:19-dgottwald@mozilla.com-priv-20080912/

Thanks!

> Note that Linux talos didn't like that build:
> FAIL: Busted: tp
> FAIL: browser frozen

Hmmm, that's weird, I can build and run the Linux version without any problems here.  Are there more detailed logs available?

(In reply to comment #203)
> I just want to confirm: This is not tab-independent, correct?  It applies to
> the whole browser.

Yes, that's correct.  The Private Browsing implementation is global to all of the Firefox windows.  According to the latest changes in the spec, the currently open windows and tabs will be closed when entering the private mode, so this may be a little less confusing.

(In reply to comment #204)
> Add a feature that will allow users to clear all items instantly for the
> sessions they had browsed during the session of mozilla. I doubt you keep up a
> history of each cookie and history of what that individual process/tab has done
> during the whole time it has been open (would be a memory hog). Sometimes a
> user is browsing and might have forgotten to set mozilla for private browsing
> instead allowing users to clear the sessions, even if it only pertains to what
> is currently open and associated cookies/history.

This is a good idea.  Currently the sanitizer (accessible from Tools > Clear Private Data) is able to clear the whole data, not the session specific data.  This is actually a sanitizer thing, and I'm not sure if it's not already filed as a bug.  Please search bugzilla and if it's not already filed, go ahead and file it, since this will need a separate bug report in order to work on.

(In reply to comment #205)
> I just took a quick look over it and it looks awesome. Great work!

Thanks! :-)

> One concern: From my quick look it didn't seem to be touching
> nsSessionStore.js. Are closed tabs remembered using your current code? (Again,
> quick look and I haven't tried it) I ask because I'm implementing "undo closed
> window" which is analogous to the "undo closed tabs" feature, and I'm trying to
> get a feel for how this might effect the time line for that, and vice versa.

According to the latest changes in the spec <https://wiki.mozilla.org/User:Mconnor/PrivateBrowsing#Session_Store>, session store will be handled in this way: when entering the private mode, all open windows and tabs will be closed, and only a single window will be left open.  The undo close tabs (and windows) list will be left untouched in private mode (i.e., the tabs and windows closed in private mode will not be added to them).  When leaving the private mode, all of the windows and tabs that the user had open before entering the private mode will be restored, and undo close tabs/windows list will get updated once again.

BTW, what's the bug# tracking the undo close windows work?  I'm personally interested in it.
Attached patch Patch (v2.4) (obsolete) — Splinter Review
This patch makes a small correction in the cache module handling to clear the memory cache when entering the private browsing mode (as per the spec).  Furthermore, it implements the session store handling as laid out in the spec.

For session store handling, nsSessionStore turns off writing the session data to disk while in private mode, and backs up the restore windows data when entering the private mode, and restores it after leaving it.  Also, it prompts the user if they want to close their current session and open a private session.  If the user says yes, then the current windows and tabs will be closed, and after leaving the private mode, they all will be restored.  If the user says no, their tabs and windows remain open and no restore takes place when leaving the private mode (the behavior of versions prior to 2.4).

This prompting is implemented in nsBrowserGlue, and can be overrided with the new browser.privatebrowsing.keep_current_session pref.  A mechanism is in place for extensions to override this UI (by handling the "private-browsing-start" notification, as I'll document in <https://wiki.mozilla.org/User:Ehsan/PrivateBrowsingForExtensions>), and the nsBrowserGlue behavior only kicks in when no extension handles this notification.  In case an extension provides its own UI here, the browser.privatebrowsing.keep_current_session will be ignored, and saving user's choice (if desired) will be the responsibility of the extension itself.

At this revision, the patch should be fully compatible with the spec.  Dao: can you submit this to the try server as well?  Thanks!

By the way, I've tested this on both Windows and Linux.  Let's see if this time Talos can run this successfully...
Attachment #338299 - Attachment is obsolete: true
Drive-by comments related to Session Restore:
* If you temporarily enter private browsing and have Firefox close your windows, what happens when you crash during private browsing? Will your original windows be restored? (AFAICT this should work already)
* When you quit while still in private browsing mode, what will be restored at the next startup with Session Restore switched on? (looks like this could be sensitive content, at least if you've decided not to close all non-private windows first)
* Instead of saving/restoring the list of closed tabs for all windows, I'd rather tag all tabs closed during private browsing mode and then just purge these when private browsing is over.
(In reply to comment #207)
> Created an attachment (id=338439) [details]
> Patch (v2.4)

https://build.mozilla.org/tryserver-builds/2008-09-13_07:23-dgottwald@mozilla.com-priv-20080913/
(In reply to comment #208)
> Drive-by comments related to Session Restore:
> * If you temporarily enter private browsing and have Firefox close your
> windows, what happens when you crash during private browsing? Will your
> original windows be restored? (AFAICT this should work already)

Yes, the original windows will be restored.

> * When you quit while still in private browsing mode, what will be restored at
> the next startup with Session Restore switched on? (looks like this could be
> sensitive content, at least if you've decided not to close all non-private
> windows first)

Again, the original windows will be restored, even if you choose not to close all non-private windows when entering the private browsing mode.  Basically, no data about the session will be saved to the disk while in private browsing mode so the contents of sessionstore.js will not change while in private mode.

> * Instead of saving/restoring the list of closed tabs for all windows, I'd
> rather tag all tabs closed during private browsing mode and then just purge
> these when private browsing is over.

What about the case where the windows/tabs are closed when entering the private mode?  Will this work in that case as well?
(In reply to comment #210)
> no data about the session will be saved to the disk while in private browsing
> mode so the contents of sessionstore.js will not change while in private mode.

You will however have to write to sessionstore.js during shutdown, otherwise Firefox will think it crashed at the next startup (unless you correctly safe the file before entering PB mode, which you currently don't). However AFAICT from skimming the code, you exit PB mode on quit-application-granted which will cause sessionstore.js to be written again with the state from when entering PB mode but (at least) updated cookies...

> What about the case where the windows/tabs are closed when entering the
> private mode?  Will this work in that case as well?

It does: You'll get the list(s) of recently closed tabs included in the browser state through getBrowserState and restore it through setBrowserState...
(In reply to comment #211)
> You will however have to write to sessionstore.js during shutdown, otherwise
> Firefox will think it crashed at the next startup (unless you correctly safe
> the file before entering PB mode, which you currently don't). However AFAICT
> from skimming the code, you exit PB mode on quit-application-granted which will
> cause sessionstore.js to be written again with the state from when entering PB
> mode but (at least) updated cookies...

So, would that be enough for Firefox to detect that it was shut down cleanly?

> It does: You'll get the list(s) of recently closed tabs included in the browser
> state through getBrowserState and restore it through setBrowserState...

I see.  Thanks for clarification.  I'll try to get around to implement your suggestion soon.
Ehsan, shouldn't the menu item be checked when you have entered the "Private Browsing" mode? At least on OS X it cannot be seen and it makes hard to see in which state you are.
(In reply to comment #213)
> Ehsan, shouldn't the menu item be checked when you have entered the "Private
> Browsing" mode? At least on OS X it cannot be seen and it makes hard to see in
> which state you are.

Yes, it should...  The thing is, I've been deferring all the UI work to bug 411929, but I guess we'd want this menu item at any rate (and if not, we can remove it in that bug) so I'll make a robust menu item which actually gets updated in response to changes in the private browsing mode in the next revision.  Thanks for bringing this into my attention, and sorry for the confusion it has caused in the QA work.
Attached patch Patch (v2.5) (obsolete) — Splinter Review
Just a small fix to the passwordmgr module, and improved the menu item in the Tools menu to fix the problem Henrik mentioned in comment 213.
Attachment #338439 - Attachment is obsolete: true
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 ID:2008070208

Patch doesn't work for me on a fresh profile on Vista (at least the patched tryserver), I get these errors (and some others):

Warning: reference to undefined property Cc['@mozilla.org/browser/privatebrowsing;1']
Source File: chrome://browser/content/browser.js
Line: 8136

Error: Cc['@mozilla.org/browser/privatebrowsing;1'] is undefined
Source File: chrome://browser/content/browser.js
Line: 8136

Error: this._privateBrowsingService is null
Source File: chrome://browser/content/browser.js
Line: 8141

Error: uncaught exception: [Exception... "Component returned failure code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE) [nsIJSCID.getService]"  nsresult: "0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)"  location: "JS frame :: chrome://browser/content/browser.js :: anonymous :: line 987"  data: no]
The latest try server build ID should be "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1pre) Gecko/20080914141735 Minefield/3.1b1pre", but you're right that it doesn't work.  I inspected this a bit, and it seems like that the nsPrivateBrowsingService.js file does not exist in the components directory at all.  I took a look at the build log <http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1221426077.1221435503.1246.gz>, and the following statements seem to suggest that the component is correctly being copied to the components directory, but in fact, it's not.

for i in /e/builds/sendchange-slave/sendchange-win32-hg/mozilla/browser/components/privatebrowsing/src/nsPrivateBrowsingService.js; do \
  dest=../../../../dist/bin/components/`basename $i`; \
  rm -f -f $dest; \
  /d/mozilla-build/python25/python /e/builds/sendchange-slave/sendchange-win32-hg/mozilla/config/Preprocessor.py -DOSTYPE=\"WINNT5.2\" -DOSARCH=WINNT -D_CRT_SECURE_NO_DEPRECATE=1 -D_CRT_NONSTDC_NO_DEPRECATE=1 -DWINVER=0x500 -D_WIN32_WINNT=0x500 -D_WIN32_IE=0x0500 -DX_DISPLAY_MISSING=1 -DMOZILLA_VERSION=\"1.9.1b1pre\" -DMOZILLA_VERSION_U=1.9.1b1pre -DHAVE_SNPRINTF=1 -D_WINDOWS=1 -D_WIN32=1 -DWIN32=1 -DXP_WIN=1 -DXP_WIN32=1 -DHW_THREADS=1 -DSTDC_HEADERS=1 -DWIN32_LEAN_AND_MEAN=1 -DNO_X11=1 -DHAVE_MMINTRIN_H=1 -DHAVE_OLEACC_IDL=1 -DHAVE_ATLBASE_H=1 -DHAVE_WPCAPI_H=1 -D_X86_=1 -DD_INO=d_ino -DMOZ_EMBEDDING_LEVEL_DEFAULT=1 -DMOZ_EMBEDDING_LEVEL_BASIC=1 -DMOZ_EMBEDDING_LEVEL_MINIMAL=1 -DMOZ_PHOENIX=1 -DMOZ_BUILD_APP=browser -DMOZ_XUL_APP=1 -DMOZ_DEFAULT_TOOLKIT=\"cairo-windows\" -DMOZ_DISTRIBUTION_ID=\"org.mozilla\" -DOJI=1 -DIBMBIDI=1 -DMOZ_VIEW_SOURCE=1 -DACCESSIBILITY=1 -DMOZ_XPINSTALL=1 -DMOZ_JSLOADER=1 -DNS_PRINTING=1 -DNS_PRINT_PREVIEW=1 -DMOZ_NO_XPCOM_OBSOLETE=1 -DMOZ_OGG=1 -DMOZ_MEDIA=1 -DMOZ_XTF=1 -DMOZ_CRASHREPORTER=1 -DMOZ_CRASHREPORTER_ENABLE_PERCENT=100 -DMOZ_MATHML=1 -DMOZ_ENABLE_CANVAS=1 -DMOZ_SVG=1 -DMOZ_UPDATE_CHANNEL=default -DMOZ_PLACES=1 -DMOZ_FEEDS=1 -DMOZ_STORAGE=1 -DMOZ_SAFE_BROWSING=1 -DMOZ_URL_CLASSIFIER=1 -DMOZ_LOGGING=1 -DMOZ_USER_DIR=\"Mozilla\" -DMOZ_ENABLE_LIBXUL=1 -DMOZ_TREE_CAIRO=1 -DHAVE_UINT64_T=1 -DMOZ_XUL=1 -DMOZ_PROFILELOCKING=1 -DMOZ_RDF=1 -DMOZ_MORKREADER=1 -DMOZ_DLL_SUFFIX=\".dll\" -DJS_THREADSAFE=1  -DNDEBUG -DTRIMMED $i > $dest; \
done


The build works correctly locally on at least Windows and Linux, and I'm not sure what I'm doing wrong which causes the try server builds not to include this component.  Can anyone see what is going wrong here?
A suggestion from another bug where patches were only working locally:
"are you creating new interfaces? If so, did you add the .xpt files to
the packaging manifests? If not, then that could be why things fail on packaged
builds but not your local builds."
The only error I can see on OS X while toggling between the states is following entry:

Error: key is null
Source File: file:///Volumes/Minefield/Minefield.app/Contents/MacOS/components/nsUrlClassifierLib.js
Line: 1173

Which is:

PROT_UrlCryptoKeyManager.prototype.unUrlSafe = function(key)
{
--> return key.replace("-", "+").replace("_", "/");
}


On Windows XP I also cannot see the given errors and don't have Vista by hand for further tests until Thursday. But what I've recognized is that there is no dialog when entering the private browsing mode. I don't think that it is expected.

Further one more question: Why the private browsing mode is not sticky? After a restart it's still unset again. I believe in environments where this mode will be used at any time (e.g. internet cafe's) it will be a hassle to have to manually switch into this mode again and again.
(In reply to comment #219)
> A suggestion from another bug where patches were only working locally:
> "are you creating new interfaces? If so, did you add the .xpt files to
> the packaging manifests? If not, then that could be why things fail on packaged
> builds but not your local builds."

Hmm, I am adding a new interface, and I think this might be the culprit.  Thanks for your suggestion.  I'll post an updated patch shortly.
(In reply to comment #220)
> The only error I can see on OS X while toggling between the states is following
> entry:
> 
> Error: key is null
> Source File:
> file:///Volumes/Minefield/Minefield.app/Contents/MacOS/components/nsUrlClassifierLib.js
> Line: 1173
> 
> Which is:
> 
> PROT_UrlCryptoKeyManager.prototype.unUrlSafe = function(key)
> {
> --> return key.replace("-", "+").replace("_", "/");
> }

I get this as well, and I haven't looked into it.  Looks like somehow this component responds to the notifications it should ignore.  Anyway this doesn't prevent the patch from functioning.

> On Windows XP I also cannot see the given errors and don't have Vista by hand
> for further tests until Thursday. But what I've recognized is that there is no
> dialog when entering the private browsing mode. I don't think that it is
> expected.

Yes, on XP no error is logged to the console, but the component and the interface are not registered (which can be verified by examining Components.classes/Components.interfaces in the error console).

> Further one more question: Why the private browsing mode is not sticky? After a
> restart it's still unset again. I believe in environments where this mode will
> be used at any time (e.g. internet cafe's) it will be a hassle to have to
> manually switch into this mode again and again.

I'm not sure if we want to preserve this across sessions (it makes it easy for someone to know if you've been in private mode last time you closed Firefox or not) but I think we can add a pref to initiate the private mode each time Firefox starts automatically.
Attached patch Patch (v2.6) (obsolete) — Splinter Review
This is the same as 2.5, only with the xpt file being added to the installer packaging manifests.  I'm not sure if this causes nsPrivateBrowsingService.js to appear in the components folder in try server builds, though.  Can someone please submit this to try server?
Attachment #338552 - Attachment is obsolete: true
Attached patch Patch (v2.6) (obsolete) — Splinter Review
Oops, please ignore the previous patch; I forgot to "hg add" the new files...  This one should be correct.
Attachment #338599 - Attachment is obsolete: true
Is there any plans to handle Flash shared objects at this point or will it be
implemented afterwards? Does
https://bugzilla.mozilla.org/show_bug.cgi?id=290456 cover this or should a
separate bug be filled for private browsing mode case?

In any case, I see two possible solutions: 
1) Clear shared objects afterwards when user leaves the private browsing mode
2) Change Flash Player security settings to temporarily block shared objects. I
think these can be adjusted by setting variables in Flash
Player\macromedia.com\support\flashplayer\sys\settings.sol
Option 2 would honor the "Don't write to disk" principle.
Ehsan, the try server build failed. Looks like your patch is bitrotted:
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1221476657.1221476875.32211.gz
(In reply to comment #225)
> Is there any plans to handle Flash shared objects at this point or will it be
> implemented afterwards? Does
> https://bugzilla.mozilla.org/show_bug.cgi?id=290456 cover this or should a
> separate bug be filled for private browsing mode case?

I think it would be better to file a new bug, since that bug covers the sanitizer module.

> In any case, I see two possible solutions: 
> 1) Clear shared objects afterwards when user leaves the private browsing mode
> 2) Change Flash Player security settings to temporarily block shared objects. I
> think these can be adjusted by setting variables in Flash
> Player\macromedia.com\support\flashplayer\sys\settings.sol
> Option 2 would honor the "Don't write to disk" principle.

Are there documents from Adobe available that show how to do the second option in a cross-platform manner?
Attached patch Patch (v2.6) (obsolete) — Splinter Review
(In reply to comment #226)
> Ehsan, the try server build failed. Looks like your patch is bitrotted:
> http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1221476657.1221476875.32211.gz

Here's the unbitrotted patch diffed against the latest pulled hg trunk.  Can you submit it again?  Thanks! :-)
Attachment #338601 - Attachment is obsolete: true
The latest patch doesn't contain the packages-static changes. You'll need to add nsPrivateBrowsingService.js to packages-static as well (see other JS components listed in that file).
Comment on attachment 338622 [details] [diff] [review]
Patch (v2.6)

(In reply to comment #229)
> The latest patch doesn't contain the packages-static changes. You'll need to
> add nsPrivateBrowsingService.js to packages-static as well (see other JS
> components listed in that file).

You're right.  New patch forthcoming.  Hope I get it right this time... :-/
Attachment #338622 - Attachment is obsolete: true
Attached patch Patch (v2.6) (obsolete) — Splinter Review
OK, this should be the right patch... ;-)
Till this gets to the tryserver... I took the liberty of uploading my own builds with this patch applied (builds are available for Windows only, sorry no cross-compiler;):

firefox.zip: https://www.yousendit.com/download/bVlCcHBLUEM1R05MWEE9PQ
.exe: http://www.yousendit.com/download/bVlCcHBFNXZUME9Ga1E9PQ

Only 100 downloads though so get to it! Great new feature, big fan of the work being done here, thanks alot.
i think that this could get better advantages of the changes we are experimenting in Places for fsync stuff, we will already have a memory table to hold places and history, and you could move using them.

Your patch has probably some issue on Places as it is, i did not read it all but if i read it correctly you don't allow adding places into moz_places returning always false to canAddURI. but then you save visits in a new table (that should be a TEMP one i think), but those visits will be orphans without a place. Also you're not ensuring to not add datas to annotations and inputhistory tables.
So you're practically removing places during private browsing, and that could make awesomebar/bookmarks and so on unusable while using it (usable only on old entries). Instead they should be correct until you're in private browsing, then clear all private data on exiting private mode.
Also what happens if a user wants to add a bookmark in private mode? probably we should still hold it since he explicitely asked to remember that site.

I have some ideas about this, so feel free to nag me on IRC if you want, sorry if i did not catch everything about your changes
Some notes:
With private browsing enabled I get the following error: Error: key is null
Source File: file:///C:/Users/Natch/Documents/mozilla_trunk/mozilla-central__release/dist/bin/components/nsUrlClassifierLib.js
Line: 1173
Also right-click bookmark a link is broken in private mode, the panel shows up on the left. I've seen other errors related to places stuff, though I haven't repro'd them reliably yet...
Depends on: 455454
(In reply to comment #234)
> Some notes:
> With private browsing enabled I get the following error: Error: key is null
> Source File:
> file:///C:/Users/Natch/Documents/mozilla_trunk/mozilla-central__release/dist/bin/components/nsUrlClassifierLib.js
> Line: 1173

I filed this as bug 455454, and attached a patch there to solve that problem.

> Also right-click bookmark a link is broken in private mode, the panel shows up
> on the left. I've seen other errors related to places stuff, though I haven't
> repro'd them reliably yet...

Is the placement of the panel the problem you're mentioning?  If so, that's where it appears normally, I think.  I guess Marco's comments (comment 233) might explain a bit of the problems you might have been experiencing in the private mode with Places.

I'll ping him on IRC for more info; in-memory tables look really interesting!
No longer depends on: 455454
(In reply to comment #231)
> Created an attachment (id=338639) [details]
> Patch (v2.6)
> 
> OK, this should be the right patch... ;-)

And here the appropriate try server builds:
https://build.mozilla.org/tryserver-builds/2008-09-15_15:23-mozilla@hskupin.info-bug248970/
Depends on: 455454
For Content Preferences this doesn't follow the Wiki as it won't update an existing pref.

> Site Specific Prefs
>
> * Nothing special for enter/exit
> * During, we will not remember new entries, but will update existing ones. In current known usage (only used for zoom in Firefox) this is not a data leak. It is possible for extension uses to leak some data here, but there's a lot of things extension authors can do wrong.
> o This effectively means that setPref() will fail if hasPref() is false. This will likely be a silent failure so as to not throw for existing callers. 

Presumably changing to something like

>    setPref: function ContentPrefService_setPref(aURI, aName, aValue) {
>      // If the pref is already set to the value, there's nothing more to do.
> +    // If we are in private browsing mode, refuse to set the pref
>      var currentValue = this.getPref(aURI, aName);
> -    if (typeof currentValue != "undefined" && currentValue == aValue)
> +    if (typeof currentValue != "undefined" ? currentValue == aValue : this._inPrivateBrowsing)
>        return;

[possibly something more readable though!]
No longer depends on: 455454
Depends on: 447648
Attached patch Patch (v2.6.1) (obsolete) — Splinter Review
(In reply to comment #237)
> For Content Preferences this doesn't follow the Wiki as it won't update an
> existing pref.

This was changed after I implemented the content prefs part.  This new patch corrects the implementation in a way similar to what you suggested.
Attachment #338639 - Attachment is obsolete: true
Probably not related but thought I'd report with the latest build from the tryserver I am getting browser crashes when I visit www.neowin.net with the tracemonkey JIT content 'enabled'.  

The official nightly does not crash.  Sorry, breakpad did not kick in either. (but that's a known, ongoing issue with flash player)
(In reply to comment #239)
> Probably not related but thought I'd report with the latest build from the
> tryserver I am getting browser crashes when I visit www.neowin.net with the
> tracemonkey JIT content 'enabled'.  
> 
> The official nightly does not crash.  Sorry, breakpad did not kick in either.
> (but that's a known, ongoing issue with flash player)

What platform are you on?  Maybe it's something changed after the latest nightly was built, can you check an hourly?  I'm not sure what may cause the crash here, and I can't reproduce it here.  Can others try this as well?
I'm on Win32 Vista HP SP1 using today's official nightly:

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b1pre) Gecko/20080916043910 Minefield/3.1b1pre Firefox/3.0 ID:20080916043910

Are there nightly's on the tryserver ?  
I was using this build when I was crashing:
https://build.mozilla.org/tryserver-builds/2008-09-15_15:23-mozilla@hskupin.info-bug248970/
(In reply to comment #241)
> I'm on Win32 Vista HP SP1 using today's official nightly:
> 
> Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b1pre)
> Gecko/20080916043910 Minefield/3.1b1pre Firefox/3.0 ID:20080916043910
> 
> Are there nightly's on the tryserver ?  
> I was using this build when I was crashing:
> https://build.mozilla.org/tryserver-builds/2008-09-15_15:23-mozilla@hskupin.info-bug248970/

I don't get a crash (or anything unusual) when visiting neowin's home page neither in private mode or usual mode on XP SP2.  I have flashplayer 9.0.124.0.  Can you try an hourly?  What about the previous try server builds on this bug?
I can try an hourly build when the next one comes out, about 20-30 mins.  

The build I noted above, was the first build I saw on the tryserver where Private mode actually worked, previous builds did not seem to enter Private-mode.  

I was crashing on NeoWin whether I was in Private mode or Normal mode.  

I was just wondering if part of the JIT/Tracemonkey stuff was maybe not in the build on the tryserver test builds ?  

If there are hourly builds for this bug could you point me to one?  I thought they were all hand-made...
No crash on NeoWin with latest 'official hourly' from here:
http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32/1221568388/

This build or course does not have the InPriv patch -
Using test build of InPriv:
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b1pre) Gecko/20080915154629 Minefield/3.1b1pre Firefox/3.0 ID:20080915154629

Bookmarking a page while in Private mode saves the bookmark.  I can see the bookmark I added after 'exit' from Private mode.
(In reply to comment #245)
> Bookmarking a page while in Private mode saves the bookmark.  I can see the
> bookmark I added after 'exit' from Private mode.

This is the expected behavior: the bookmark should be saved, but no visit to the bookmarked site should be recorded.  See <https://wiki.mozilla.org/Firefox3.1/PrivateBrowsing/FunctionalSpec> for more info.
(In reply to comment #243)
> I can try an hourly build when the next one comes out, about 20-30 mins.  
> 
> The build I noted above, was the first build I saw on the tryserver where
> Private mode actually worked, previous builds did not seem to enter
> Private-mode.  

Anyway, since you mention you got crashes both in private and normal mode, it makes sense to look into previous builds, I think.  If the crashes stop at a particular build, we can assume it's something I did between those revisions of the patch, and tracking it would be far simpler.

FWIW, I tested this both using Henrik's build, and my local build on Vista, neither crashed NeoWin...

> I was just wondering if part of the JIT/Tracemonkey stuff was maybe not in the
> build on the tryserver test builds ?  

I don't think so.  And there's nothing I do in the patch which touches any Tracemonkey stuff either.

> If there are hourly builds for this bug could you point me to one?  I thought
> they were all hand-made...

No, I meant the trunk hourly builds.  Sorry for the confusion.
(In reply to comment #246)
> This is the expected behavior: the bookmark should be saved, but no visit to
> the bookmarked site should be recorded.

interesting, you don't have a place and canAddURI return false (so you should not be able to add one), but using insertBookmark canAddURI is not checked, and a new place is added. I guess if this was done by design or we miss a check in getUrlIdFor.
I don't know what else to do, I don't crash with the 'official' builds.  I guess we will see what happens when it finally makes it into the M-C official builds.
When entering Private Mode the cursor is not focused in the LocationBar.  I would think this would be nice, as in normal op's when opening a 'blank' window the cursor is focued there.
(In reply to comment #248)
> interesting, you don't have a place and canAddURI return false (so you should
> not be able to add one), but using insertBookmark canAddURI is not checked, and
> a new place is added. I guess if this was done by design or we miss a check in
> getUrlIdFor.

I carefully made sure that getUrlIdFor is not affected by the private browsing mode.  As far as I know, there are three places where getUrlIdFor is called with auto create mode turned on:

 * when creating bookmarks (which we should allow in private mode)
 * when creating favicons (which we shouldn't allow in private mode, and my patch handles it now)
 * when creating annotations.

I'm not really sure about what Places annotations mean, and where (in the UI) they are used, so I was not sure whether I need to turn off creating them in the private mode.  Any hints on that would be greatly appreciated.
(In reply to comment #249)
> I don't know what else to do, I don't crash with the 'official' builds.  I
> guess we will see what happens when it finally makes it into the M-C official
> builds.

Are you using a new profile or an already existing one?  If the latter, can you try with a clean profile?

Also, can you disable your plugins one by one and see if the crashes happen with all of them disabled?

(In reply to comment #250)
> When entering Private Mode the cursor is not focused in the LocationBar.  I
> would think this would be nice, as in normal op's when opening a 'blank' window
> the cursor is focued there.

Hmmm, I think this should be handled by browser.js automatically when a new window is opened, but I may be wrong.  I'll need to check this, but at least I can confirm this problem here as well.  :-)
I've been testing with an existing profile.  Just created a new one, no addons, no bookmarks.  Still crashes, and continues to crash when loading www.neowin.net even with all the plugins 'disabled'. 

JIT content is not enabled by 'default' yet.. are you sure you have it set to 'true' ? 

about:config
javascript.options.jit.content Value=true

I won't have time for anymore testing today..will catch up tomorrow.
(In reply to comment #253)
> I've been testing with an existing profile.  Just created a new one, no addons,
> no bookmarks.  Still crashes, and continues to crash when loading
> www.neowin.net even with all the plugins 'disabled'. 
> 
> JIT content is not enabled by 'default' yet.. are you sure you have it set to
> 'true' ? 

Yes.

I'm CCing some TraceMonkey gurus to see if they can help (read comment 239 onwards).
I get the same crash on www.neowin.net (Windows XP).
Jim and Ria, could one of you try to get the stack trace with Windbg?

http://developer.mozilla.org/en/How_to_get_a_stacktrace_with_WinDbg
Attached patch Patch (v2.7) (obsolete) — Splinter Review
This test fixes a problem with half-renaming a pref, and adds a unit test for cookies, according to <https://wiki.mozilla.org/Firefox3.1/PrivateBrowsing/TestPlan#Unit_Tests>.

One thing I ran into problems with was running the nsICookieManager2.cookieExists method through the test.  cookieExists causes xpcshell to crash.  I'm not sure, but I think the crash may be because of the fact that the C++ implementation of cookieExists actually expects an nsCookie parameter, but since this method is not marked as noscript, I suspected that there may be something I'm missing.  Anyone can shed some light into this?

I had to disable the is_cookie_available2 checks for now because of the crash.
Attachment #338836 - Attachment is obsolete: true
(In reply to comment #257)
> One thing I ran into problems with was running the
> nsICookieManager2.cookieExists method through the test.  cookieExists causes
> xpcshell to crash.  I'm not sure, but I think the crash may be because of the
> fact that the C++ implementation of cookieExists actually expects an nsCookie
> parameter, but since this method is not marked as noscript, I suspected that
> there may be something I'm missing.  Anyone can shed some light into this?
That would indeed crash.  Can you file a bug on that please - the cookie service shouldn't assume it's an nsCookie.
Depends on: 455598
(In reply to comment #258)
> That would indeed crash.  Can you file a bug on that please - the cookie
> service shouldn't assume it's an nsCookie.

Done: bug 455598.  I attached a patch on that bug to fix this issue.  With that patch applied, the is_cookie_available2 checks in my test can be enabled, and the test correctly passes.
(In reply to comment #256)
> Jim and Ria, could one of you try to get the stack trace with Windbg?
> 
> http://developer.mozilla.org/en/How_to_get_a_stacktrace_with_WinDbg

Wouldn't you know it, its not crashing today...if it comes up again, I'll try and get a trace with WinDbg.
Attached patch Patch (v2.8) (obsolete) — Splinter Review
What's new:

 * Now that the fix for bug 455598 landed, I enabled the nsICookieManager2.cookieExists tests in the cookies unit test.  Now, the cookie test follows the test plan exactly.
 * The previous patch had an oversight in the cookies unit test which caused it not to test the private browsing mode at all (duh!); fixed in this patch.
 * Added the unit test for content prefs, according to the test plan.
 * Added the browser UI test for passwordmgr, according to the test plan.
 * I tried to address the latest changes regarding passwordmgr in the functional spec as well (to enable filling of the HTTP auth dialog) and I noticed a strange problem:

If, while initiating the private browsing mode, I choose to keep the current session open, everything's cool.  If, however, I choose to save and close the session, the auth dialog turns up empty.  After enabling signon.debug, I noticed a couple of these messages which indicated that the username and password pair could not be decrypted:

PwMgr mozStorage: Failed to decrypt string: MDIEEPgAAAAAAAAAAAAAAAAAAAEwFAYIKoZIhvcNAwcECKifd14qd4yXBAhGB3/m2HDvtA== (NS_ERROR_FAILURE)

I tried running the below code in the error console, and it also failed with NS_ERROR_FAILURE:

Components.classes["@mozilla.org/security/sdr;1"].
getService(Components.interfaces.nsISecretDecoderRing).
decryptString("MDIEEPgAAAAAAAAAAAAAAAAAAAEwFAYIKoZIhvcNAwcECKifd14qd4yXBAhGB3/m2HDvtA==")

I also tried opening the password manager, and it was empty as well.  I'm not sure how saving and closing the session could affect the SecretDecoderRing service.  Does anyone have any ideas?
Attachment #338946 - Attachment is obsolete: true
You're most likely running into bug 437904. I'll try to investigate what's going on there; shouldn't be related to anything you're doing.
(In reply to comment #262)
> You're most likely running into bug 437904. I'll try to investigate what's
> going on there; shouldn't be related to anything you're doing.

Looks related (I'm clearing the authenticated sessions upon entering the private browsing mode) but in this case, whether or not the user decides to save and close her current session affects the results...

I'd appreciate if you can take a deeper look and see if you can find something which my eyes have missed.
(In reply to comment #262)
> You're most likely running into bug 437904. I'll try to investigate what's
> going on there; shouldn't be related to anything you're doing.

That's true. Even without the patch running the above command in the Error Console throws this exception.

Ehsan, I've started a new try server build. Please have a look at the tinderbox site when it will be finished. I'm offline now.

http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTry
(In reply to comment #264)
> That's true. Even without the patch running the above command in the Error
> Console throws this exception.

No, that doesn't happen here.  I guess the exception you're observing is caused by the fact that the encrypted string can't be decrypted with the key on your machine.  You can obtain an encrypted string for use in that code by looking into the passwords sqlite db, I guess (or reproducing the failure I observed with signon.debug turned on).

> Ehsan, I've started a new try server build. Please have a look at the tinderbox
> site when it will be finished. I'm offline now.
> 
> http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTry

Thanks, I'll try to keep an eye on it if I'm awake by the time it finishes (it's 2:30AM here), or check it tomorrow morning.  :-)
Linux build was unsuccessful, but I doubt it had anything to do with the patch <http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1221688098.1221688924.20353.gz>.  Windows talos was orange with the error "previous cycle still running", and I'm not sure what that means (seems like many other patches caused an orange win32 talos with the same error message) <http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1221696517.1221696581.9936.gz>.
(In reply to comment #251)
> I'm not really sure about what Places annotations mean, and where (in the UI)
> they are used, so I was not sure whether I need to turn off creating them in
> the private mode.  Any hints on that would be greatly appreciated.

Annotations are notes that can be added by us or an extension developer to urls or bookmarks, so it can contain any data, personal or uri related data too (it depends on how the implementer uses it).
The main problem here is with page annotations, mainly because if an extension adds a page annotation during private browsing and the place_id does not exist we would probably try to add anno for place_id = -1 since you don't have a place.
You should probably early return in nsAnnotationService::SetPageAnnotation while in private browsing mode for now, or check for place id validity later (better handling could be done though).

about moz_inputhistory you should early return in nsNavHistory::AutoCompleteFeedback se we will not save what urls user has choosen in the awesomebar after searching for a certain text.
A few notes:
Download Manager is broken in private browsing mode, although files to get downloaded the DM cannot (and perhaps should not, but it should be notified of PB) tell you the progress of the download, also I stumbled across these errors:
Error: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsITransfer.onProgressChange64]"  nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)"  location: "JS frame :: chrome://global/content/contentAreaUtils.js :: anonymous :: line 138"  data: no]
Source File: chrome://global/content/contentAreaUtils.js
Line: 138
When completing the download:
Error: download is null
Source File: chrome://mozapps/content/downloads/DownloadProgressListener.js
Line: 78
When hitting the [x] on the download:
Error: uncaught exception: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIDownloadManager.cancelDownload]"  nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)"  location: "JS frame :: chrome://mozapps/content/downloads/downloads.js :: cancelDownload :: line 181"  data: no]

I was using private browsing in "keep my current session" mode, if that helps at all.
(In reply to comment #269)
Correction: Either I'll get that or the DM works fine, but on exit the DM retains the download data, from the wiki it seems the downloads should be reloaded from pre-PB mode. Also not the downloads aren't recorded in history (correctly).
Hi,

I am a student looking into writing and or collaborating on unit tests for the patches you guys are submitting for private browsing mode.

Can someone direct me to or simply list the aspect(s) of Private Browsing mode that need testing (unit tests).

For example, Ehsan, I see you have listed some cases & testing areas - here 
https://wiki.mozilla.org/User:Ehsan/PrivateBrowsingTests#Unit_Tests

Are they in order of priority?

I'd appreciate any feedback, 
Thanks.

- AaronMT
(In reply to comment #270)
> (In reply to comment #269)
> Correction: Either I'll get that or the DM works fine, but on exit the DM
> retains the download data, from the wiki it seems the downloads should be
> reloaded from pre-PB mode. Also not the downloads aren't recorded in history
> (correctly).

Can you reproduce this in a clean profile?  I can't reproduce it here.  What happens is that while the file is being downloaded, its progress should appear in the download manager, and when it's finished, it should be removed from the download manager.  I don't get errors on the console either...  Anyway, the functional spec has laid out new requirements for the download manager, so its implementation will change in the future revisions of my patch.
(In reply to comment #271)
Aaron: thanks for your interest in helping out! :-)

The current test plan is being written by mconnor <https://wiki.mozilla.org/Firefox3.1/PrivateBrowsing/TestPlan>.  From the tests listed there, there are two which are not completed yet (I've marked the completed ones with a star next to their title).  I've a half-baked test for the unit tests section of the password manager, so that leaves out the history unit tests, which you can pick up to work on.

Another thing which needs testing is the private browsing service itself.  There is also an API for extensions which needs unit tests <https://wiki.mozilla.org/User:Ehsan/PrivateBrowsingForExtensions>.  It would be great if you can add a test plan for testing the APIs to <https://wiki.mozilla.org/Firefox3.1/PrivateBrowsing/TestPlan>, and start working on them as well.

Please let me know if you need any help (preferably via email, since this bug is already too huge).

Thanks!
(In reply to comment #268)
> Annotations are notes that can be added by us or an extension developer to urls
> or bookmarks, so it can contain any data, personal or uri related data too (it
> depends on how the implementer uses it).

Thanks for the explanation.

> The main problem here is with page annotations, mainly because if an extension
> adds a page annotation during private browsing and the place_id does not exist
> we would probably try to add anno for place_id = -1 since you don't have a
> place.
> You should probably early return in nsAnnotationService::SetPageAnnotation
> while in private browsing mode for now, or check for place id validity later
> (better handling could be done though).

What about the typed versions of SetPageAnnotation?  They can be called through the nsIAnnotationService interface.  I guess a good solution here would be to change nsAnnotationService::GetPlaceIdForURI to ignore the auto create parameter if we're in the private browsing mode, and handle that case gracefully in all of its callers with auto create mode set.  Do you agree?

> about moz_inputhistory you should early return in
> nsNavHistory::AutoCompleteFeedback se we will not save what urls user has
> choosen in the awesomebar after searching for a certain text.

Thanks for mentioning this.  The next revision of the patch will include this change.
(In reply to comment #208)
> * Instead of saving/restoring the list of closed tabs for all windows, I'd
> rather tag all tabs closed during private browsing mode and then just purge
> these when private browsing is over.

Simon: I tried to give this approach a shot, but it won't work, since we trim the _closedTabs array by max_tabs_undo all the time, so it's possible for some items in that array to get lost while in private mode, so after leaving this mode, the entire list as it was prior to entering this mode cannot be restored.
(In reply to comment #275)
> it's possible for some items in that array to get lost while in private mode,

Actually, I wouldn't expect all the closed tabs to be still around after having used PB mode in a pre-existing window, in the same way as you don't expect the very same tabs to be around when you exit PB mode (you don't restore those to their original content, if the user didn't choose to close the original windows, do you?). So reducing or even completely clearing the list of recently closed tabs seems alright to me - otherwise you could reopen a tab in PB mode and reopen that same tab again after exiting PB mode, which strikes me as kind of odd.
(In reply to comment #276)
> Actually, I wouldn't expect all the closed tabs to be still around after having
> used PB mode in a pre-existing window, in the same way as you don't expect the
> very same tabs to be around when you exit PB mode (you don't restore those to
> their original content, if the user didn't choose to close the original
> windows, do you?).

No, we don't, but come to think of it, maybe we should?  Mconnor: what do you think?

> So reducing or even completely clearing the list of recently
> closed tabs seems alright to me - otherwise you could reopen a tab in PB mode
> and reopen that same tab again after exiting PB mode, which strikes me as kind
> of odd.

What about the case where user chooses to open a new window to for the private browsing mode, and save and close her current session?
(In reply to comment #274)
> What about the typed versions of SetPageAnnotation?  They can be called through
> the nsIAnnotationService interface.  I guess a good solution here would be to
> change nsAnnotationService::GetPlaceIdForURI to ignore the auto create
> parameter if we're in the private browsing mode, and handle that case
> gracefully in all of its callers with auto create mode set.  Do you agree?

Well i could agree, notice that Shawn is trying to land the fsync stuff in these days (he will probably retry today), means that if things appear good (no build boxes failures and timings) you could move posting your visits/places into the temp tables, in that case the auto create change would not be useful because you could have a place already (hwv if i read code correctly we are not using the autoCreate actually).
But i suppose that GetPlaceIdForURI is still a good place to start hacking, in future you could change it to return -1 if the page is "private" and handle that case in type specific methods, and for now you can do the same, return -1 if in private mode and handle that.
(In reply to comment #277)
> What about the case where user chooses to open a new window to for the private
> browsing mode, and save and close her current session?

Not closing any tabs in a pre-existing window won't modify the list of recently closed tabs, so you'll get the expected behavior. And closing/restoring all windows before entering/after exiting PB mode will restore the list of recently closed tabs with the windows, again as expected.
(In reply to comment #278)
> Well i could agree, notice that Shawn is trying to land the fsync stuff in
> these days (he will probably retry today), means that if things appear good (no
> build boxes failures and timings) you could move posting your visits/places
> into the temp tables, in that case the auto create change would not be useful

I guess in that case, we could in fact not turns off anything in nsNavHistory, and just have everything use the temp tables, instead of the on-disk ones, right?

By the way, I'm not sure what APIs can be used to work with temp in-memory tables.  Is it something along the lines of doing a "PRAGMA temp_store=MEMORY;", creating temporary counterpart tables for each of the places tables, duplicating the data in on-disk tables on these new tables and having all the code use the new tables?

> because you could have a place already (hwv if i read code correctly we are not
> using the autoCreate actually).

autoCreate is set to true by default, so the function calls which don't specify it explicitly get the true default value.

> But i suppose that GetPlaceIdForURI is still a good place to start hacking, in
> future you could change it to return -1 if the page is "private" and handle
> that case in type specific methods, and for now you can do the same, return -1
> if in private mode and handle that.

I didn't get the difference between the "now" and "future" cases as you explained above, but that's what I'm going to do.  :-)
(In reply to comment #280)
> (In reply to comment #278)
> I guess in that case, we could in fact not turns off anything in nsNavHistory,
> and just have everything use the temp tables, instead of the on-disk ones,
> right?

by default we will add to the temp tables, and then sync on a timer or on bookmark-insert or on browser-exit to the disk, so 2 possibilities:
- don't sync to disk when we are in private browsing mode, clear temp tables on exit. Cons: this is complex to handle if we are allowing the user to create bookmarks while in private browsing mode since we would need to sync them down to disk.
- in private browsing mode mark places and visits with a "private" column in the temp table, on sync skip marked places/visits (this can be done easily modifying our triggers). Cons: needs temp table change and manage tables with different columns (disk table does not need a "private" column).

> By the way, I'm not sure what APIs can be used to work with temp in-memory
> tables.  Is it something along the lines of doing a "PRAGMA
> temp_store=MEMORY;", creating temporary counterpart tables for each of the
> places tables, duplicating the data in on-disk tables on these new tables and
> having all the code use the new tables?

using a temp table is simply done creating it like CREATE TEMP TABLE table_name, temp_store is not really useful (hwv we are already using it).
fsync stuff hwv inserts and updates in temp tables by default (yes we are partitioning places and visits table between disk and memory).

> I didn't get the difference between the "now" and "future" cases as you
> explained above, but that's what I'm going to do.  :-)

"now" is before fsync stuff, when you can't have a place. "future" is when you could have a place saved in a temp table. However imho you should start having "placeholders changes" in every place where you'll need to manager private browsing mode (that's what you're doing), impl can change later based on where we are moving to.
Attached patch Patch (v2.9) (obsolete) — Splinter Review
What's new in this revision:

 * Handled nsAnnotationService in private browsing mode, to make sure it doesn't store any annotations.
 * Handled nsNavHistory::AutoCompleteFeedback in private browsing mode, to make sure that user's choice in awesomebar does not get stored.
 * Corrected the behavior of the session store component on exit/restart.  Previously, if the user chose to restart the browser (for example, by installing an add-on) in private browsing mode, or closed the browser (provided that browser.startup.page is set to 3) without leaving the private mode, the private session was stored on disk, and would later be restored.  With this revision, this shouldn't happen any more, and the user's non-private session should be restored, as expected.
Attachment #339117 - Attachment is obsolete: true
(In reply to comment #281)
> - in private browsing mode mark places and visits with a "private" column in
> the temp table, on sync skip marked places/visits (this can be done easily
> modifying our triggers). Cons: needs temp table change and manage tables with
> different columns (disk table does not need a "private" column).

I think the second option can be used even without a new column, by using a new temp table and joining it against moz_places.

> using a temp table is simply done creating it like CREATE TEMP TABLE
> table_name, temp_store is not really useful (hwv we are already using it).
> fsync stuff hwv inserts and updates in temp tables by default (yes we are
> partitioning places and visits table between disk and memory).

Hmm, this page seems to suggest that temp tables are written to the disk while the DB connection is open <http://www.sqlite.org/tempfiles.html>.  Ideally we wouldn't want to do that.  According to the docs, temp_store might be able to save us there...
(In reply to comment #283)
> (In reply to comment #281)
> I think the second option can be used even without a new column, by using a new
> temp table and joining it against moz_places.

we can't, it's already quite difficult having one temp table, having two would be a pain. that's because sqlite does not support indices on unions or views, so we have to use very specialized queries to read from both (disk and memory).

> Hmm, this page seems to suggest that temp tables are written to the disk while
> the DB connection is open <http://www.sqlite.org/tempfiles.html>.  Ideally we
> wouldn't want to do that.  According to the docs, temp_store might be able to
> save us there...

temp_store is memory in Places
After playing with a couple of builds I went back to official nightly builds.  Today I discovered that new URL's were not being shown when entered into the Location bar.  

Examination with History->Show all history, did show the URL but the visit count was zero.  I got to looking at other history of my normal visited sites and found that none were being 'counted as visited' anymore. 

I renamed Places.sqlite to places.sqlite.old and restarted the browser to find that my bookmarks were still there ?  My history was indeed gone, and is now being 'counted as visisted and incrementing' as it should.  

Bug or ?  Almost seems that I was stuck in PB mode.  Honestly I cannot recall if I properly exited PB mode before going back to official nightly's.
Jim: have you tried to reproduce this?  Your comment was a bit vague, and I need some STR in order to try to debug this.
No, I just discovered this at the time of my comment #286 above.  I had to leave for work, so will be tomorrow, or wednesday before I have time to install the PB tryserver build and see if things break again.  

Can you explain what seems to vague?  

The issue seems to be that none of the sites I visit are being shown in the Library with a 'visit count'.  The URL is there but the visit count is zero.  As long as the count stays at zero of course it won't show in the Location bar when doing a search.
I tried to reproduce what Jim is seeing in Comment 285 and was not able to do so, but I was testing with a fresh profile. Here is what I did:

1. Downloaded the latest private browsing tryserver build
2. Created a fresh profile.
3. Surfed 4-5 sites than invoked PB mode. I left the other browsing session open by selecting that preference.
4. Browsed 4-5 sites in PB mode.
5. Exited the browser and started up the latest trunk nightly using the same profile.
6. Browsed to the sites that in regular mode and revisited. The visit count incremented correctly and the sites showed up in the location bar.

What I did not do is use an existing profile that had already been run on the trunk to see what happens, I can do that next, since that is probably closer to what Jim was doing.
Yes I was using a months old profile that I normally keep history for 90 days. 

Thanks for testing Marcia.  While your at it, could you also try:

Open the browser, enter PB mode, then close and re-install a current official nightly build?  I'm really suspecting that something was left 'locked' in my places.sqlite file, thus leaving the browser 'thinking' it was still in PB Mode.

I have SQLite browser, but I'm not sure what to look for, and will be tomorrow before I can even do that.
Initiated a new try server build with patch v2.9. Should be available in an hour or so.
I tested the scenario that Jim outlined in Comment 289 using a Windows Vista VM. Again, here are the steps I followed:

1. Downloaded and installed the latest tryserver build.
2. Entered PB mode and surfed a few sites. Exited FF still in PB mode.
3. Ran the uninstaller and uninstalled the tryserver build.
4. Installed the latest trunk nightly (Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b1pre) Gecko/20080922032349 Minefield/3.1b1pre) and ran with the same profile.

Result: Everything seemed okay as far as being able to access sites and the visit counts registered appropriately. The sites that I visited in PB mode **did** show up in history when I relaunched using the trunk, which I am not certain is expected or not - will this present an issue for individuals going back and forth between builds once the feature is launched?
(In reply to comment #291)
> Result: Everything seemed okay as far as being able to access sites and the
> visit counts registered appropriately. The sites that I visited in PB mode
> **did** show up in history when I relaunched using the trunk, which I am not
> certain is expected or not - will this present an issue for individuals going
> back and forth between builds once the feature is launched?

This is not expected, and it may be fixed in the 2.9 version of the patch.  Could you please re-run the test steps with the builds Henrik provided in comment 292?  It would be nice if you can make a backup copy of your places.sqlite before entering the private browsing mode, and after leaving it, so that if this problem persists, I can use them in order to debug this.

Also, before step 4, can you check and make sure that the visited sites do not show up in the PB-enabled build itself?  They should appear there as well if they'd appear in the latest nightly build.

Thanks!
Just a couple notes.  This build is still not playing nice with TM JIT.Content enabled.  Also its 'crashy' at times, with no breakpad being fired.  Neowin.net continues to crash with JIT.Content enabled as well. 

It does appear from bouncing back and forth from this build, to an 'official' nightly build is OK, I don't see anything I browse in PB Mode appearing back in the trunk nightly build.  

It also appears that early testing with PB builds did corrupt/break my places.sqlite file as it continues to log sites/url's visited with a visit count of zero.  A new places.sqlite file does appear to function normally, even bouncing back and forth from PB builds to 'official' nightly builds. 

I will try and get a debug trace tomorrow when I have some time for the crash a neowin.net
Here is the debug trace.
1.  PB build is running 'normal' - not in private mode
2.  visit: neowin.net
3.  after the site is just about done loading 'crash' 
4.  JIT.content is 'enabled' - note this does not crash on official nightly's

===============================================================================
*** wait with pending attach
Symbol search path is: C:\websymbols;C:\localsymbols
Executable search path is: 
ModLoad: 00870000 00887000   C:\Program Files\Minefield\firefox.exe
ModLoad: 77bd0000 77cf7000   C:\Windows\system32\ntdll.dll
ModLoad: 77a60000 77b3b000   C:\Windows\system32\kernel32.dll
ModLoad: 64de0000 656d3000   C:\Program Files\Minefield\xul.dll
ModLoad: 69580000 695e3000   C:\Program Files\Minefield\sqlite3.dll
ModLoad: 73520000 735bb000   C:\Windows\WinSxS\x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.1434_none_d08b6002442c891f\MSVCR80.dll
ModLoad: 779b0000 77a5a000   C:\Windows\system32\msvcrt.dll
ModLoad: 684a0000 68554000   C:\Program Files\Minefield\js3250.dll
ModLoad: 10000000 10029000   C:\Program Files\Minefield\nspr4.dll
ModLoad: 76510000 765d6000   C:\Windows\system32\ADVAPI32.dll
ModLoad: 77640000 77702000   C:\Windows\system32\RPCRT4.dll
ModLoad: 73ad0000 73ad7000   C:\Windows\system32\WSOCK32.dll
ModLoad: 77370000 7739d000   C:\Windows\system32\WS2_32.dll
ModLoad: 764d0000 764d6000   C:\Windows\system32\NSI.dll
ModLoad: 74c10000 74c42000   C:\Windows\system32\WINMM.dll
ModLoad: 767c0000 7685d000   C:\Windows\system32\USER32.dll
ModLoad: 77d00000 77d4b000   C:\Windows\system32\GDI32.dll
ModLoad: 765e0000 76724000   C:\Windows\system32\ole32.dll
ModLoad: 77b40000 77bcd000   C:\Windows\system32\OLEAUT32.dll
ModLoad: 75e20000 75e59000   C:\Windows\system32\OLEACC.dll
ModLoad: 00080000 00098000   C:\Program Files\Minefield\smime3.dll
ModLoad: 00630000 006da000   C:\Program Files\Minefield\nss3.dll
ModLoad: 00110000 00124000   C:\Program Files\Minefield\nssutil3.dll
ModLoad: 00130000 00137000   C:\Program Files\Minefield\plc4.dll
ModLoad: 00150000 00157000   C:\Program Files\Minefield\plds4.dll
ModLoad: 00170000 00190000   C:\Program Files\Minefield\ssl3.dll
ModLoad: 76860000 7736f000   C:\Windows\system32\SHELL32.dll
ModLoad: 77710000 77768000   C:\Windows\system32\SHLWAPI.dll
ModLoad: 75de0000 75de8000   C:\Windows\system32\VERSION.dll
ModLoad: 74640000 74682000   C:\Windows\system32\WINSPOOL.DRV
ModLoad: 76450000 764c3000   C:\Windows\system32\COMDLG32.dll
ModLoad: 75890000 75a2e000   C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.6001.18000_none_5cdbaa5a083979cc\COMCTL32.dll
ModLoad: 764e0000 764fe000   C:\Windows\system32\IMM32.dll
ModLoad: 77520000 775e8000   C:\Windows\system32\MSCTF.dll
ModLoad: 75740000 75745000   C:\Windows\system32\MSIMG32.dll
ModLoad: 77930000 779ad000   C:\Windows\system32\USP10.dll
ModLoad: 735c0000 73647000   C:\Windows\WinSxS\x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.1434_none_d08b6002442c891f\MSVCP80.dll
ModLoad: 74320000 74327000   C:\Program Files\Minefield\xpcom.dll
ModLoad: 77d50000 77d59000   C:\Windows\system32\LPK.DLL
ModLoad: 74fb0000 74fef000   C:\Windows\system32\uxtheme.dll
ModLoad: 74c00000 74c0c000   C:\Windows\system32\dwmapi.dll
ModLoad: 777a0000 7792a000   C:\Windows\system32\SETUPAPI.dll
ModLoad: 76300000 7631e000   C:\Windows\system32\USERENV.dll
ModLoad: 762e0000 762f4000   C:\Windows\system32\Secur32.dll
ModLoad: 74af0000 74bab000   C:\Windows\system32\PROPSYS.dll
ModLoad: 76730000 767b4000   C:\Windows\system32\CLBCatQ.DLL
ModLoad: 72260000 72274000   C:\Program Files\Minefield\components\browserdirprovider.dll
ModLoad: 756d0000 7570b000   C:\Windows\system32\mswsock.dll
ModLoad: 75370000 75375000   C:\Windows\System32\wshtcpip.dll
ModLoad: 75ab0000 75ac9000   C:\Windows\system32\iphlpapi.dll
ModLoad: 75a70000 75aa5000   C:\Windows\system32\dhcpcsvc.DLL
ModLoad: 75bd0000 75bfc000   C:\Windows\system32\DNSAPI.dll
ModLoad: 75a60000 75a67000   C:\Windows\system32\WINNSI.DLL
ModLoad: 75a30000 75a51000   C:\Windows\system32\dhcpcsvc6.DLL
ModLoad: 74bf0000 74bff000   C:\Windows\system32\NLAapi.dll
ModLoad: 73a00000 73a0f000   C:\Windows\system32\napinsp.dll
ModLoad: 73030000 73042000   C:\Windows\system32\pnrpnsp.dll
ModLoad: 73430000 73438000   C:\Windows\System32\winrnr.dll
ModLoad: 773a0000 773ea000   C:\Windows\system32\WLDAP32.dll
ModLoad: 76440000 76447000   C:\Windows\system32\PSAPI.DLL
ModLoad: 75730000 75735000   C:\Windows\System32\wship6.dll
ModLoad: 75780000 757a1000   C:\Windows\system32\NTMARTA.DLL
ModLoad: 75bb0000 75bc1000   C:\Windows\system32\SAMLIB.dll
ModLoad: 73ab0000 73ab6000   C:\Windows\system32\rasadhlp.dll
ModLoad: 74690000 74743000   C:\Windows\system32\WindowsCodecs.dll
ModLoad: 75450000 7548b000   C:\Windows\system32\rsaenh.dll
ModLoad: 02810000 02835000   C:\Program Files\Minefield\softokn3.dll
ModLoad: 02840000 02858000   C:\Program Files\Minefield\nssdbm3.dll
ModLoad: 02b20000 02b59000   C:\Program Files\Minefield\freebl3.dll
ModLoad: 02b60000 02ba9000   C:\Program Files\Minefield\nssckbi.dll
ModLoad: 6e860000 6e8a0000   C:\Program Files\Minefield\components\brwsrcmp.dll
ModLoad: 6dce0000 6dd42000   C:\Windows\system32\mscms.dll
ModLoad: 71220000 71327000   C:\Windows\system32\shdocvw.dll
ModLoad: 64500000 64965000   C:\Windows\system32\Macromed\Flash\NPSWF32.dll
ModLoad: 77d60000 77e30000   C:\Windows\system32\WININET.dll
ModLoad: 76500000 76503000   C:\Windows\system32\Normaliz.dll
ModLoad: 775f0000 77635000   C:\Windows\system32\iertutil.dll
ModLoad: 75fe0000 760d1000   C:\Windows\system32\CRYPT32.dll
ModLoad: 75fa0000 75fb2000   C:\Windows\system32\MSASN1.dll
ModLoad: 773f0000 77519000   C:\Windows\system32\urlmon.dll
ModLoad: 71530000 71560000   C:\Windows\system32\mlang.dll
ModLoad: 747a0000 747cf000   C:\Windows\system32\wdmaud.drv
ModLoad: 748e0000 748e4000   C:\Windows\system32\ksuser.dll
ModLoad: 74ac0000 74ae7000   C:\Windows\system32\MMDevAPI.DLL
ModLoad: 74e00000 74e07000   C:\Windows\system32\AVRT.dll
ModLoad: 752a0000 752cd000   C:\Windows\system32\WINTRUST.dll
ModLoad: 77770000 77799000   C:\Windows\system32\imagehlp.dll
ModLoad: 74610000 74631000   C:\Windows\system32\AUDIOSES.DLL
ModLoad: 744b0000 74516000   C:\Windows\system32\audioeng.dll
ModLoad: 74600000 74609000   C:\Windows\system32\msacm32.drv
ModLoad: 742c0000 742d4000   C:\Windows\system32\MSACM32.dll
ModLoad: 74490000 74497000   C:\Windows\system32\midimap.dll
ModLoad: 75860000 75867000   C:\Windows\system32\credssp.dll
ModLoad: 754c0000 75504000   C:\Windows\system32\schannel.dll
ModLoad: 76120000 76195000   C:\Windows\system32\NETAPI32.dll
(c0.ab0): Break instruction exception - code 80000003 (first chance)
eax=7ffab000 ebx=00000000 ecx=00000000 edx=77c5d094 esi=00000000 edi=00000000
eip=77c17dfe esp=0c5dfac0 ebp=0c5dfaec iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\system32\ntdll.dll - 
ntdll!DbgBreakPoint:
77c17dfe cc              int     3
0:016> g
(c0.26c): Illegal instruction - code c000001d (first chance)
(c0.26c): Illegal instruction - code c000001d (!!! second chance !!!)
eax=00005d61 ebx=ffffffff ecx=00000000 edx=80000000 esi=00000901 edi=6854cb04
eip=6853c035 esp=002aabd8 ebp=002aabe0 iopl=0         nv up ei pl nz na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00010206
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Minefield\js3250.dll - 
js3250!JS_XDRFindClassById+0x29c65:
6853c035 f20f2c0424      cvttsd2si eax,mmword ptr [esp] ss:0023:002aabd8=fff8000000000000

===============================================================================
Vista HP SP1 
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b1pre) Gecko/20080922155248 Minefield/3.1b1pre Firefox/3.0 ID:20080922155248
This build seems to only enter PB Mode one time, then I cannot enter PB mode again without closing/restarting the browser.
Sorry for the spam.
Turning 'off' JIT.Content allows entry/exit to PB mode multiple times.
Now, I've discovered if one of the tabs is a link/tab to say this bug, on exit from PB Mode the tab opens/restores as 'untitled'.  Using the location bar trying to open the link does nothing but open another 'untitled' tab.  i.e., the link cannot be visited unless you close/restart the browser. 

After several attempts of trying to discover the issue the browser got to the point it would not start up at all without crashing.

Reloaded the official nightly, and clicking 'restore from last session', all tabs loaded as expected without crashing. 

Something is really not playing nice somewhere, but its beyond me at this point.  

I really feel this needs more eyes/testers at this point as its very unstable at the moment.  I assume when the try-server builds are made, its from a current pull form hg ??  i.e., all the latest/greatest patches etc...
Thanks for your help in testing this patch, Jim!

(In reply to comment #294)
> Just a couple notes.  This build is still not playing nice with TM JIT.Content
> enabled.  Also its 'crashy' at times, with no breakpad being fired.  Neowin.net
> continues to crash with JIT.Content enabled as well. 

Hmmm, I still can't reproduce this.  Marcia:  I really need some help here with testing this.  Can you spend some time testing this on Vista, please?

> It does appear from bouncing back and forth from this build, to an 'official'
> nightly build is OK, I don't see anything I browse in PB Mode appearing back in
> the trunk nightly build.  

I assume it's because of the latest work on moz_inputhistory and the annotations service.  Glad to hear it's fixed.

> It also appears that early testing with PB builds did corrupt/break my
> places.sqlite file as it continues to log sites/url's visited with a visit
> count of zero.  A new places.sqlite file does appear to function normally, even
> bouncing back and forth from PB builds to 'official' nightly builds. 

Sorry to hear about the corrupted places.sqlite.  If it doesn't contain any personal data, I'd be happy to take a look at it if possible.

(In reply to comment #295)
> Here is the debug trace.
> 1.  PB build is running 'normal' - not in private mode
> 2.  visit: neowin.net
> 3.  after the site is just about done loading 'crash' 
> 4.  JIT.content is 'enabled' - note this does not crash on official nightly's

It would be great if you can set up WinDbg to use Microsoft and Mozilla's symbol servers: <http://developer.mozilla.org/en/Using_the_Mozilla_symbol_server>.  Try server builds symbols are available at <http://build.mozilla.org/tryserver-symbols>.

(In reply to comment #297)
> Sorry for the spam.
> Turning 'off' JIT.Content allows entry/exit to PB mode multiple times.
> Now, I've discovered if one of the tabs is a link/tab to say this bug, on exit
> from PB Mode the tab opens/restores as 'untitled'.  Using the location bar
> trying to open the link does nothing but open another 'untitled' tab.  i.e.,
> the link cannot be visited unless you close/restart the browser. 

Something's really wrong with how the PB patch interacts with tracemonkey, I guess.  Maybe using the symbols for this build, we can start investigating this issue.  By the way, do you see these problems in a fresh profile as well?  What about other platforms (if you have tested)?

> I really feel this needs more eyes/testers at this point as its very unstable
> at the moment.  I assume when the try-server builds are made, its from a
> current pull form hg ??  i.e., all the latest/greatest patches etc...

Yes, they are.  Marcia: can you help with testing this as well?
(In reply to comment #298)
> <http://developer.mozilla.org/en/Using_the_Mozilla_symbol_server>.  Try server
> builds symbols are available at <http://build.mozilla.org/tryserver-symbols>.

For the try server symbols I get a 403 (forbidden). But as Ted told me it could be that this folder is not browsable. But if anyone have issues accessing the symbols please give a note.
I retested my steps from Comment 291 with the tryserver build that Henrik generated in Comment 292 using my Win Vista VM:

1. Downloaded and installed the latest tryserver build.
2. Entered PB mode and surfed a few sites. Exited FF still in PB mode.
3. Ran the uninstaller and uninstalled the tryserver build.
4. Installed the latest trunk nightly (Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b1pre) Gecko/20080923111052 Minefield/3.1b1pre) and ran with the
same profile.

Result: When I launched the trunk build I could not see the sites that I launched in PB mode, so everything works as expected in this regard.

I also enabled JIT right from the getgo and visited the site that Jim is crashing on but I was not able to reproduce that crash.

I also wondered if Jim was running with a FF profile that was possibly migrated from FF2?
I did not migrate any profiles from FF2, in fact I've never used FF2 as I'm a nightly 'junkie', and enjoy testing/breaking the new builds.

I crash at neowin.net using a new profile as well.  What's totally baffling me, is why it does not crash using 3.1b1pre trunk nightly's.  

Marcia, did you try going into and out of PB Mode several times with JIT.content enabled?  Seems I could only get it to do one cycle of in/out of PB mode.  

I will try to download the try-server symbols tomorrow and run debug.  I'm not real versed in using WinDbg but will see what I can do.  

My earlier post did have both Firefox sysmbols and windows sysmbols loaded with my trace.. I didn't know there was different symbol set for the try-server.
I cannot get neowin.net to fail this morning to get a trace in windbg.  There must be some doggy ad or something coming around once in ahwile, but that still does not fully explain why it works OK with official nightly's.

There is still something up when you have an https page open during normal browsisng and then enter PB mode choosing to 'save and close' then on exit of PBmode the https site will show as 'untitled'.  I found that by being persistant enough on clicking the 'go' arrow it will eventually load.  Must be something funky in the way its saving or not saving https sites on entry to PB Mode.

I'll be waiting for the next iteration and try some more testing.
(In reply to comment #302)
> There is still something up when you have an https page open during normal
> browsisng and then enter PB mode choosing to 'save and close' then on exit of
> PBmode the https site will show as 'untitled'.  I found that by being
> persistant enough on clicking the 'go' arrow it will eventually load.  Must be
> something funky in the way its saving or not saving https sites on entry to PB
> Mode.

Simon, Dietrich: Could this be because of the check being performed in nsSessionStore._checkPrivacyLevel?  Maybe the SSL page status does not get saved as part of the usual session store data.  If that is the case, is there anything to be done without compromising users' privacy to solve this?
(In reply to comment #303)
The very same code path works fine when PB mode isn't involved. AFAICT what silently(!) fails is the call to |browser.webNavigation.gotoIndex| at <http://mxr.mozilla.org/mozilla-central/source/browser/components/sessionstore/src/nsSessionStore.js#1627>. The tab's history seems to be correctly restored, though, which you can verify by navigating in the tab before entering PB mode and by right-clicking the Back button after exiting it.

Could your code changes subtly interact with a browser's webNavigation to that end? Or could you try to restore the browsing state while PB mode isn't completely over and we're in an inconsistent state overall?
BTW: For the latest try-server build, sessionStorage seems to be broken. At least my patch for bug 339445 suddenly throws a NS_ERROR_DOM_SECURITY_ERR for nsIDOMStorage::GetLength (which it doesn't in the latest nightly build). Could this be due to your patch?
(In reply to comment #301)
[snip]

I just retested the build on Win Vista and navigated in and out of PB mode several times with JIT enabled. When I then launched a fresh trunk build everything worked as expected - all the sites I browsed while in PB mode did not show and it cycled back and forth fine during my testing.
> 
> Marcia, did you try going into and out of PB Mode several times with
> JIT.content enabled?  Seems I could only get it to do one cycle of in/out of PB
> mode.  
>
Depends on: 456884
(In reply to comment #304)
> (In reply to comment #303)
> The very same code path works fine when PB mode isn't involved. AFAICT what
> silently(!) fails is the call to |browser.webNavigation.gotoIndex| at
> <http://mxr.mozilla.org/mozilla-central/source/browser/components/sessionstore/src/nsSessionStore.js#1627>.

Hmmm, we're eating the exception there, right?  Looks like an exception is expected there.  Would you please explain why it's been implemented like that, and what kinds of exceptions have been expected?

> The tab's history seems to be correctly restored, though, which you can verify
> by navigating in the tab before entering PB mode and by right-clicking the Back
> button after exiting it.

Yes, this seems to be the case.

> Could your code changes subtly interact with a browser's webNavigation to that
> end?

Not in any way that I know, but I could be wrong...

Could it be a problem with the way I'm closing all current windows, and opening a new one?  I've noticed that session store would use that first opened window it it's still open while exiting the private mode, so maybe there's something wrong with that window?  Here's the relevant code from the patch:

+  _closeAllWindows: function PBS__closeAllWindows() {
+    let windowMediator = Cc["@mozilla.org/appshell/window-mediator;1"].
+                         getService(Ci.nsIWindowMediator);
+    let windowsEnum = windowMediator.getEnumerator("navigator:browser");
+    
+    this._closeAllWindowsInternal(windowsEnum);
+  },
+
+  _closeAllWindowsInternal: function PBS__closeAllWindowsInternal(windowsEnum) {
+    if (windowsEnum.hasMoreElements()) {
+      let window = windowsEnum.getNext();
+      this._closeAllWindowsInternal(windowsEnum);
+      window.close();
+    }
+  },
+
+  _openSingleWindow: function PBS__openSingleWindow() {
+    let windowWatcher = Cc["@mozilla.org/embedcomp/window-watcher;1"].
+                        getService(Ci.nsIWindowWatcher);
+    windowWatcher.openWindow(null, "chrome://browser/content/browser.xul",
+                             null, "chrome,all,resizable=yes,dialog=no", null);
+  },



> Or could you try to restore the browsing state while PB mode isn't
> completely over and we're in an inconsistent state overall?

Well, in the private browsing service, I first notify the observers about the exit event, and then call nsSessionStore.setBrowserState to restore the session, so the private browsing mode should be well finished by that time.

(In reply to comment #305)
> BTW: For the latest try-server build, sessionStorage seems to be broken. At
> least my patch for bug 339445 suddenly throws a NS_ERROR_DOM_SECURITY_ERR for
> nsIDOMStorage::GetLength (which it doesn't in the latest nightly build). Could
> this be due to your patch?

Yes.  nsDOMStorage::GetLength calls nsDOMStorage::CacheStoragePermissions which itself calls nsDOMStorage::CanUseStorage which returns false in private browsing mode, and hence the NS_ERROR_DOM_SECURITY_ERR exception.  Basically, you can't use the dom storage service in the private browsing mode.  This was suggested by Dave Camp in comment 172.

I'm not sure what can we do instead of handling the code in bug 339445 after it gets landed in private mode as well.  So, I'm adding it as a dependency here.
Blocks: 339445
(In reply to comment #307)
> Hmmm, we're eating the exception there, right?

No, there's no exception at all - the call just (silently) fails. The only exceptions we are expecting come from passing an invalid index to gotoIndex which actually shouldn't be happening anymore, at all. Could you step into gotoIndex with a debugger and see what's actually happening in there?

> Could it be a problem with the way I'm closing all current windows, and
> opening a new one?

No, as when restoring the saved session is all handled by SessionStore itself. (BTW: Why don't you just loop over the iterator in _closeAllWindows instead of recursively calling _closeAllWindowsInternal? |while| to the rescue?)

> (In reply to comment #305)
> Yes. nsDOMStorage::GetLength calls nsDOMStorage::CacheStoragePermissions which
> itself calls nsDOMStorage::CanUseStorage which returns false in private
> browsing mode, and hence the NS_ERROR_DOM_SECURITY_ERR exception.

Problem is: I got the same error right after startup (i.e. before even touching PB mode at all). Bug?

> I'm not sure what can we do instead of handling the code in bug 339445 after it
> gets landed in private mode as well.

What about throwing or returning |null| from getSessionStorageForURI while in PB mode instead so that sessionStorage consumers can bail out sooner rather than later?
(In reply to comment #308)
> (In reply to comment #307)
> > Hmmm, we're eating the exception there, right?
> 
> No, there's no exception at all - the call just (silently) fails. The only
> exceptions we are expecting come from passing an invalid index to gotoIndex
> which actually shouldn't be happening anymore, at all. Could you step into
> gotoIndex with a debugger and see what's actually happening in there?

Sure, I'll do that.

> > Could it be a problem with the way I'm closing all current windows, and
> > opening a new one?
> 
> No, as when restoring the saved session is all handled by SessionStore itself.
> (BTW: Why don't you just loop over the iterator in _closeAllWindows instead of
> recursively calling _closeAllWindowsInternal? |while| to the rescue?)

Ni special reason, I just thought it would look better visually to close the windows in reverse order, and nsISimpleEnumerator doesn't support reverse iterations...

> Problem is: I got the same error right after startup (i.e. before even touching
> PB mode at all). Bug?

Oh, in that case, yes it's probably a bug.  I need to investigate it.

> What about throwing or returning |null| from getSessionStorageForURI while in
> PB mode instead so that sessionStorage consumers can bail out sooner rather
> than later?

That won't cover consumers which call getSessionStorageForURI prior to entering the private mode, would it?
(In reply to comment #309)
> Sure, I'll do that.

Thanks!

> nsISimpleEnumerator doesn't support reverse iterations...

In that case, please at least add a comment to that end.

> That won't cover consumers which call getSessionStorageForURI prior to entering
> the private mode, would it?

Sure, so s/instead/additionally/.

BTW: Please don't forget to add unit tests for the SessionStore changes as well.
(In reply to comment #310)
> (In reply to comment #309)
> > Sure, I'll do that.
> 
> Thanks!

OK, I couldn't reproduce this bug on my machine.  I opened AMO and navigated to a few places on AMO, an then entered the private mode, loaded some sites and switched back to normal mode, and the AMO page was restored correctly.

Simon, do you have a STR which I can use here?

I took a look inside nsDocShell::GotoIndex anyway, and I noticed that the reason of the silent failure (which is intentional BTW) is <http://hg.mozilla.org/mozilla-central/file/5aaa5bcc6024/docshell/base/nsDocShell.cpp#l2860>.  Now, nsDocShell::IsNavigationAllowed should return false if we're printing (or in print preview mode), or if mFiredUnloadEvent is true (which logically should be the case here).  Is there any docshell guru I can ping who may be able to help me understand why this is failing?

> In that case, please at least add a comment to that end.

Sure.

> Sure, so s/instead/additionally/.

Dave: can I safely modify nsDocShell::GetSessionStorageForUI to call nsDOMStorage::CanUseStorage to determine if session storage is available or not, and return an error if it isn't?

> BTW: Please don't forget to add unit tests for the SessionStore changes as
> well.

Mconnor has written a test plan which includes SessionStore.  You can check it at <https://wiki.mozilla.org/Firefox3.1/PrivateBrowsing/TestPlan#Session_store>.  I will eventually implement all these tests before submitting the patch for review.
(In reply to comment #311)
> OK, I couldn't reproduce this bug on my machine.  I opened AMO and navigated to
> a few places on AMO, an then entered the private mode, loaded some sites and
> switched back to normal mode, and the AMO page was restored correctly.

Please ignore this, I made a mistake, but I was able to reproduce this easily paying more attention; just forgot to update my comment before submitting it.
Attached patch Patch (v2.10) (obsolete) — Splinter Review
This patch completes the password manager tests (I've added a few things to the test plan mconnor wrote: <https://wiki.mozilla.org/index.php?title=Firefox3.1%2FPrivateBrowsing%2FTestPlan&diff=110155&oldid=110125>), and adds the test for authenticated sessions.  The history tests are being done by Aaron Train (a Seneca college student) and I'm working on the session store tests right now.
Attachment #339569 - Attachment is obsolete: true
Depends on: 457110
Attached patch [for review] cookie v1 (obsolete) — Splinter Review
The cookies module patch with a unit test, ready for review.
Attachment #341413 - Flags: superreview?(bzbarsky)
Attachment #341413 - Flags: review?(dwitte)
Attached patch [for review] contentprefs v1 (obsolete) — Splinter Review
The contentprefs module patch with a unit test, ready for review.
Attachment #341414 - Flags: review?(mconnor)
Attached patch [for review] passwordmgr v1 (obsolete) — Splinter Review
The passwordmgr module patch with a unit test, ready for review.
Attachment #341415 - Flags: review?(dolske)
Attached patch [for review] httpauth v1 (obsolete) — Splinter Review
The httpauth module unit test, ready for review.  The code responsible for clearing the HTTP authenticated sessions when entering and exiting the private browsing mode lives in <https://bugzilla.mozilla.org/attachment.cgi?id=340369&action=diff#a/browser/components/privatebrowsing/src/nsPrivateBrowsingService.js_sec1>, inside the _onPrivateBrowsingModeChanged function.
Attachment #341418 - Flags: superreview?(bzbarsky)
Attachment #341418 - Flags: review?(bzbarsky)
Comment on attachment 341418 [details] [diff] [review]
[for review] httpauth v1

I don't really know the auth stuff well enough to tell whether this test is testing the right things.
Attachment #341418 - Flags: superreview?(cbiesinger)
Attachment #341418 - Flags: superreview?(bzbarsky)
Attachment #341418 - Flags: review?(cbiesinger)
Attachment #341418 - Flags: review?(bzbarsky)
Comment on attachment 341413 [details] [diff] [review]
[for review] cookie v1

>+    if (!nsCRT::strcmp(aData, NS_LITERAL_STRING("enter").get())) {

if (NS_LITERAL_STRING("enter").Equals(aData)) ?

Similar for exit.

>+      // backup the existing in-memory DB

I would tend to just make both tables pointers instead of direct members, and then all that needs to happen on enter is:

  mBackupHostTable = mHostTable;
  mHostTable = new ...;
  mHostTable->Init();

and on exit:

  mHostTable = mBackupHostTable;
  mBackupHostTable = nsnull;

You'd have to change various other code, of course.

Check with dwitte as to which he prefers here; for now I haven't looked in detail at the copying functions.
Attached patch [for review] cookie v2 (obsolete) — Splinter Review
(In reply to comment #320)
>>+    if (!nsCRT::strcmp(aData, NS_LITERAL_STRING("enter").get())) {
>
> if (NS_LITERAL_STRING("enter").Equals(aData)) ?
>
> Similar for exit.

Done.

>>+      // backup the existing in-memory DB
>
> I would tend to just make both tables pointers instead of direct members, and
> then all that needs to happen on enter is:
>
>  mBackupHostTable = mHostTable;
>  mHostTable = new ...;
>  mHostTable->Init();
>
> and on exit:
>
>  mHostTable = mBackupHostTable;
>  mBackupHostTable = nsnull;
>
> You'd have to change various other code, of course.
>
> Check with dwitte as to which he prefers here; for now I haven't looked in
> detail at the copying functions.

Hmmm, actually it seems a lot cleaner than the current copying I'm doing...  Daniel, what do you think?
Attachment #341413 - Attachment is obsolete: true
Attachment #341481 - Flags: superreview?(bzbarsky)
Attachment #341481 - Flags: review?(dwitte)
Attachment #341413 - Flags: superreview?(bzbarsky)
Attachment #341413 - Flags: review?(dwitte)
Attached patch [for review] cookie v2 (obsolete) — Splinter Review
I just realized that the previous version of the patch had a typo which prevented it from getting compiled (duh!).  This one should be fine.
Attachment #341481 - Attachment is obsolete: true
Attachment #341517 - Flags: superreview?(bzbarsky)
Attachment #341517 - Flags: review?(dwitte)
Attachment #341481 - Flags: superreview?(bzbarsky)
Attachment #341481 - Flags: review?(dwitte)
Attachment #341517 - Flags: superreview?(bzbarsky)
Attachment #341517 - Flags: review?(dwitte)
Attachment #341517 - Flags: review-
Comment on attachment 341517 [details] [diff] [review]
[for review] cookie v2

>+    if (NS_SUCCEEDED(mObserverService->NotifyObservers(inPrivateBrowsing,
>+                                                       "private-browsing-query",
>+                                                       nsnull))) {

split this into two lines, |rv = ...| and |if (NS_SUCCEEDED(rv))| please.

i also think the private-browsing-query thing is a terribly roundabout way of doing things, and i'd much prefer a service somewhere we can directly call...

>+  } else if (!strcmp(aTopic, "browser:private-browsing")) {
>+    if (NS_LITERAL_STRING("enter").Equals(aData)) {

aData.Equals(...) please (and below).

>+      // backup the existing in-memory DB

agreed with bz - go with member pointers here, and just twiddle them around to "back up" your db and reinitialize and so on. then you won't need to change the hashtable code in nsCookieService.h, either. r- pending that, since i'd like to look at that code once you've written it...
Comment on attachment 341415 [details] [diff] [review]
[for review] passwordmgr v1

General nit: patches are easier to review with more context in the diffs... See http://developer.mozilla.org/en/Mercurial_FAQ#Configuration (specifically, the "-U 8" part).

>--- a/toolkit/components/passwordmgr/src/nsLoginManager.js
...
>+        var inPrivateBrowsing = Cc["@mozilla.org/supports-PRBool;1"].
>+                                createInstance(Ci.nsISupportsPRBool);
>+        this._observerService.notifyObservers(inPrivateBrowsing,
>+                                              "private-browsing-query", null);
>+        this._inPrivateBrowsing = inPrivateBrowsing.data;

This seems wrong to me. I hope we're not using this pattern in other places.

Like dwitte said, this really seems like it should ask a service for the initial state. Something like:

  var PBS = Cc["mumble"].getService(Ci.mumble);
  this._inPrivateBrowsing = PBS.inPrivateBrowsing;

>+            } else if (topic == "browser:private-browsing") {
>+                if (data == "enter")
>+                    this._pwmgr._inPrivateBrowsing = true;
>+                else if (data == "exit")
>+                    this._pwmgr._inPrivateBrowsing = false;

Would be nice to add a debug line here...

  this._pwmgr.log("Private browsing mode: " + data);

>-        var autofillForm = this._prefBranch.getBoolPref("autofillForms");
>+        var autofillForm = !this._inPrivateBrowsing &&
>+          this._prefBranch.getBoolPref("autofillForms");

Nit: second line should be indented to align with "!".

>--- a/toolkit/components/passwordmgr/src/nsLoginManagerPrompter.js
>-function LoginManagerPrompter() {}
>+function LoginManagerPrompter() {
>+    this.init();
>+}

There's already an init() method in this object, so I'm surprised this works (or at least doesn't break something else).

But I don't think there's a need for an observer here.. A new prompter is created every time the authentication prompt is shown (ie, it's short lived), so the observer isn't likely to be triggered anyway. A simple helper function / getter, called when needed, is sufficient, and eliminated the need for an observer and more init() code...

  get _inPrivateBrowsing : function () {
    var PBS = Cc[xxx].getService(Ci.xxx);
    return PBS.inPrivateBrowsing;
  }

>-        if (!ok || !checkBox.value || !hostname)
>+        if (!ok || !checkBox.value || !hostname || this._inPrivateBrowsing)

I think this extra check isn't needed (ditto for the other places like this). The checkbox should end up hidden by the previous code, so checkbox.value should always be false.

>--- a/toolkit/components/passwordmgr/test/Makefile.in
...
>+		test_bug_248970.html \

Hmm, let's call this test_privbrowsing.html so it's more obvious what it does (ditto for subtests).

>+++ b/toolkit/components/passwordmgr/test/subtst_bug_248970_1.html
...
>+<h2>Subtest 1</h2>

Add a brief comment in here saying what the test is trying to do (eg, trigger a save/change pw prompt).

>+++ b/toolkit/components/passwordmgr/test/subtst_bug_248970_3.html
...
>+function submitForm() {
>+  setTimeout(function(){ form.submit(); }, 100);
>+}

Why the setTimeout()?


>+++ b/toolkit/components/passwordmgr/test/subtst_bug_248970_4.html
...
>+function submitForm() {
...
>+  var result = pwmgr.autoCompleteSearch("", null, userField);
>+  if (result instanceof Ci.nsIAutoCompleteResult &&
>+      result.matchCount == 1) {
>+    userField.focus();
>+    userField.value = result.getValueAt(0);
>+    userField.blur();
>+  }
>+
>+  setTimeout(function(){ form.submit(); }, 100);
>+}

Seems like this code should be the top test (test_privbrowsing.html), not in the subtest. Why call pwmgr.autoCompleteSearch? That's kind of a hacky API, and I'd like to remove it eventually... Look at test_basic_form_autocomplete.html for how to drive the autocomplete dropdown without this.


>+++ b/toolkit/components/passwordmgr/test/test_bug_248970.html

I'm ambivalent about having this as a separate test, verses rolling these tests into existing tests (test_notifications.html and test_basic_form_autocomplete.html). It kinda makes sense to test this as an independent feature, but also kinda sucks to duplicate the code/infrastructure of those tests. Maybe we should look at spinning out some of the existing stuff into common files (eg the existing pwmgr_common.js, or a new file like notification_common.js). This could be done today in a separate bug...

>+var prefix = "http://test2.example.com/tests/toolkit/components/passwordmgr/test/";

See the tests in bug 403420 to avoid hardcoding this path.

>+function get_PBSvc() {
...
>+  if (PBS_CID in Components.classes &&
>+      "nsIPrivateBrowsingService" in Components.interfaces) {
>+    try {
>+      _PBSvc = Components.classes[PBS_CID].
>+               getService(Components.interfaces.nsIPrivateBrowsingService);
>+      if (_PBSvc) {
>+        var observer = {
...

I'm not sure what this is all for. You shouldn't need to check if the service Cc/Ci exists (just let it throw+fail if it doesn't), and the observer doesn't seem to do anything useful?
Attachment #341415 - Flags: review?(dolske) → review-
Depends on: 458498
Attached patch [for review] cookie v3 (obsolete) — Splinter Review
(In reply to comment #323)
> (From update of attachment 341517 [details] [diff] [review])
> >+    if (NS_SUCCEEDED(mObserverService->NotifyObservers(inPrivateBrowsing,
> >+                                                       "private-browsing-query",
> >+                                                       nsnull))) {
> 
> split this into two lines, |rv = ...| and |if (NS_SUCCEEDED(rv))| please.

Done.

> i also think the private-browsing-query thing is a terribly roundabout way of
> doing things, and i'd much prefer a service somewhere we can directly call...

I agree, but that means we should put nsIPrivateBrowsingService.idl somewhere in the build stack before netwerk, and I'm not sure if there's an appropriate place for this (and if yes, where).  In fact, this is a hacky workaround against exactly this point.

> >+  } else if (!strcmp(aTopic, "browser:private-browsing")) {
> >+    if (NS_LITERAL_STRING("enter").Equals(aData)) {
>
> aData.Equals(...) please (and below).

Hmmm, aData is a const PRUnichar*, so I'm not sure what you mean here.

> >+      // backup the existing in-memory DB
>
> agreed with bz - go with member pointers here, and just twiddle them around to
> "back up" your db and reinitialize and so on. then you won't need to change the
> hashtable code in nsCookieService.h, either. r- pending that, since i'd like to
> look at that code once you've written it...

OK, here's the new code.  The backup and restore functions are gone now, and the changes to the hash key have been reverted as well.
Attachment #341517 - Attachment is obsolete: true
Attachment #341795 - Flags: superreview?(bzbarsky)
Attachment #341795 - Flags: review?(dwitte)
Attached patch [for review] contentprefs v2 (obsolete) — Splinter Review
New patch with more context, and using nsIPrivateBrowsingService to get the initial status of the private browsing mode.
Attachment #341414 - Attachment is obsolete: true
Attachment #341797 - Flags: review?(mconnor)
Attachment #341414 - Flags: review?(mconnor)
Depends on: 458598
Why couldn't nsIPrivateBrowsingService.idl just live in netwerk?
(In reply to comment #327)
> Why couldn't nsIPrivateBrowsingService.idl just live in netwerk?

This is a good idea.  All the modules edited in this patch depend on netwerk, so it won't cause any dependency problems.  I'll move nsIPrivateBrowsingService.idl to netwerk/base/public and will update various modules to query the private browsing service directly.  The "private-browsing-query" notification can also be discarded with this change.
Yeah, netwerk/base/public is a dumping ground for all kinds of random stuff anyway...
Depends on: 458954
No longer depends on: 458498
Attachment #341795 - Flags: superreview?(bzbarsky)
Attachment #341795 - Flags: review?(dwitte)
Comment on attachment 341795 [details] [diff] [review]
[for review] cookie v3

punting review, pending a new patch with nsIPBS.
Attachment #341418 - Flags: superreview?(cbiesinger)
Attachment #341418 - Flags: review?(cbiesinger)
Attachment #341418 - Flags: review-
Comment on attachment 341418 [details] [diff] [review]
[for review] httpauth v1

all the private browsing code lives outside of necko, right? then so should the test.
Where does/will the documentation for the private browsing feature reside?  Is it worth mentioning there that our private browsing mode has no effect on the operation of ISP or firewall that might be caching our browsing activities?
(In reply to comment #332)
> Where does/will the documentation for the private browsing feature reside?  Is
> it worth mentioning there that our private browsing mode has no effect on the
> operation of ISP or firewall that might be caching our browsing activities?

Yes, that should definitely be mentioned.  I'm not sure if there's a plan to include a specific page for it on mozilla.com, or is it just the support article, but I'm CCing David Tenser to make sure this will happen for the SUMO article at least.
Keywords: user-doc-needed
Attached file history unit test (obsolete) —
Attachment #342643 - Flags: review?(ehsan.akhgari)
Attachment #342643 - Attachment description: [for review] history v1 → [for review] history unit test
(In reply to comment #335)
I imagine this has to be uploaded as a patch (hg diff -u8)... see http://developer.mozilla.org/En/Creating_a_patch
Comment on attachment 342643 [details]
history unit test

Thanks Aaron for the unit test.  I've included this in my mq, and it'll be in the next revision of the patch which I'm attaching to the bug right now...
Attachment #342643 - Attachment description: [for review] history unit test → history unit test
Attachment #342643 - Flags: review?(ehsan.akhgari) → review+
Attached patch Patch (v2.11) (obsolete) — Splinter Review
This makes all of the splitted patches I'd posted so far obsolete.  I'll attach new versions of those patches shortly.
Attachment #340369 - Attachment is obsolete: true
Attachment #341415 - Attachment is obsolete: true
Attachment #341418 - Attachment is obsolete: true
Attachment #341795 - Attachment is obsolete: true
Attachment #341797 - Attachment is obsolete: true
Attachment #341797 - Flags: review?(mconnor)
(In reply to comment #338)
> Created an attachment (id=342650) [details]
> Patch (v2.11)

This patch should be applied on top of my latest patches for bug 458954 and bug 457110.

Here is the list of changes in this patch:

1. nsIPrivateBrowsingService.idl was moved to netwerk/base/public.
2. All of the modules now call the private browsing service directly to get the current status upon startup.
3. The private-browsing-query notification, which was merely added for the purpose of not having to depend on nsIPrivateBrowsingService, was removed.
4. The private-browsing-requested was added.  This notification is sent before entering or leaving the private mode.  Extensions and modules have a chance to cancel this mode if they desire (download manager handles this notification).
5. The bug which caused session store to save the private session to disk when restarting from within the private mode is fixed.
6. The bug which caused session store to think the browser has crashed if the user tired to close the browser while still in private mode is fixed.
7. Latest fixes as per review comments for the cookies and password manager modules are included in this patch.  (The changes requested to passwordmgr unit tests are not included yet).
8. Aaron's unit test for places is now included, making the places module ready for review.
9. The cookies and httpauth module tests are now splitted.
10. The latest functional spec for the downloads manager has been implemented in both the back-end and the front-end UI.
Attached patch [for review] contentprefs v3 (obsolete) — Splinter Review
Latest version of the contentprefs module patch.
Attachment #342661 - Flags: review?(mconnor)
Attached patch [for review] cookie v4 (obsolete) — Splinter Review
Latest version of the cookie patch, with the changes requested in comment 330, and the rest of the review comments.
Attachment #342662 - Flags: superreview?(bzbarsky)
Attachment #342662 - Flags: review?(dwitte)
Attached patch [for review] places v1 (obsolete) — Splinter Review
The places module patch with a unit test, ready for review.
Attachment #342664 - Flags: review?(dietrich)
(In reply to comment #342)
> Created an attachment (id=342664) [details]
> [for review] places v1

well this is not a review, but some comment on a couple doubts i have, hope it could be useful...

 {
   PRInt64 placeId;
   nsresult rv = GetPlaceIdForURI(aURI, &placeId);
   NS_ENSURE_SUCCESS(rv, rv);
 
+  if (placeId == -1) // we're in private browsing mode
+    return NS_OK;
+

i'm not really sure about this, placeId == -1 imho should throw and not be considered as valid, i fear this could hide other possible bugs or db corruptions. Maybe you should directly check if we are in privatebrowsing mode through an internal helper method.

-#define PLACES_SCHEMA_VERSION 7
+#define PLACES_SCHEMA_VERSION 8

you should not upgrade schema version if you're not providing a migrateV8Up function with the table changes from v7

+    NS_ENSURE_SUCCESS(rv, rv);
+  }
+
+  // moz_privatebrowsinghistory
+  rv = mDBConn->TableExists(NS_LITERAL_CSTRING("moz_privatebrowsinghistory"), &tableExists);
+  NS_ENSURE_SUCCESS(rv, rv);
+  if (!tableExists) {
+    rv = mDBConn->ExecuteSimpleSQL(CREATE_MOZ_PRIVATEBROWSINGHISTORY);
     NS_ENSURE_SUCCESS(rv, rv);
   }

why do you need to create a table to hold only 1 value that is inserted and removed sometimes, and you don't need to do searches or manipulation of this data? would not be better a pref or another kind of persistent storage (since i suppose a user could close the browser in private mode and we should rememeber that value to restart in private browsing mode?)?

+  rv = mDBConn->CreateStatement(NS_LITERAL_CSTRING(
+      "SELECT last_entered FROM moz_privatebrowsinghistory "
+      "ORDER BY last_entered DESC"),
+    getter_AddRefs(mDBGetLastPBEnterDate));
+  NS_ENSURE_SUCCESS(rv, rv);
+
+  rv = mDBConn->CreateStatement(NS_LITERAL_CSTRING(
+      "INSERT INTO moz_privatebrowsinghistory "
+      "(last_entered) "
+      "VALUES(?1)"),
+    getter_AddRefs(mDBSetLastPBEnterDate));
+  NS_ENSURE_SUCCESS(rv, rv);
+
+  rv = mDBConn->CreateStatement(NS_LITERAL_CSTRING(
+      "DELETE FROM moz_privatebrowsinghistory"),
+    getter_AddRefs(mDBRemoveLastPBEnterDate));
   NS_ENSURE_SUCCESS(rv, rv);

caching statements at startup is good for statements executed often, these are executed a few times, or never in a session, if you're going to hold the table you should probably not cache these since it would regress startup time and not provide enough perf gain. And you could directly use executeSimpleSQL for the delete.
Comment on attachment 342662 [details] [diff] [review]
[for review] cookie v4

>+  } else if (!strcmp(aTopic, "browser:private-browsing")) {
>+    if (NS_LITERAL_STRING("enter").Equals(aData)) {
>+      // backup the existing in-memory DB
>+      PRBool ok = PR_TRUE;
>+      if (!mPrivateHostTable.IsInitialized())
>+        ok = mPrivateHostTable.Init();
+      if (ok) {
+        mHostTable = &mPrivateHostTable;
+        mCookieCount = mHostTable->Count();
+        NotifyChanged(nsnull, NS_LITERAL_STRING("cleared").get());
+      }

just write this as |if (mPrivateHostTable.IsInitialized() || mPrivateHostTable.Init())| and skip the |ok| bit.

>+    } else if (NS_LITERAL_STRING("exit").Equals(aData)) {
>+      // open the connection to the on-disk DB
>+      InitDB();
>+      // restore the in-memory DB as it was prior to private browsing
>+      if (mPrivateHostTable.IsInitialized())
>+        mPrivateHostTable.Clear();
>+      mHostTable = &mDefaultHostTable;
>+      mCookieCount = mHostTable->Count();
>+     // continue to use both on-disk and in-memory DB

hmm. you fire a "cleared cookie list" notification when entering PB, but nothing on exit - so the cookiemanager and such will still show PB cookies, no? but we can't just fire a clear here either, since it needs to be repopulated. seems like we either need to teach CM about PB notifications so it can do its own thing, or we could have PBS fire a profile-related notification ("profile-do-change" perhaps?) to force reloads? not sure what other effects that might have...

also, missing indentation space on the last line there.

r=me on the cookieservice bits, but we still need a solution for cookiemanager.
Attachment #342662 - Flags: review?(dwitte) → review+
Attached patch [for review] cookie v5 (obsolete) — Splinter Review
(In reply to comment #344)
> hmm. you fire a "cleared cookie list" notification when entering PB, but
> nothing on exit - so the cookiemanager and such will still show PB cookies, no?
> but we can't just fire a clear here either, since it needs to be repopulated.
> seems like we either need to teach CM about PB notifications so it can do its
> own thing, or we could have PBS fire a profile-related notification
> ("profile-do-change" perhaps?) to force reloads? not sure what other effects
> that might have...

I think a better solution would be to add a new cookie-changed notification type, called "reload", which forces clients to reload their cookie list, and fire it when exiting the private mode.

I have implemented this, and wrote a test for it as well.  I've also changed the cookies dialog, and I'm requesting a review from Gavin on those parts.

Other comments and nits are also fixed.
Attachment #342662 - Attachment is obsolete: true
Attachment #342686 - Flags: superreview?(bzbarsky)
Attachment #342686 - Flags: review?(gavin.sharp)
Attachment #342686 - Flags: review?(dwitte)
Attachment #342662 - Flags: superreview?(bzbarsky)
Attached patch [for review] places v2 (obsolete) — Splinter Review
(In reply to comment #343)
> well this is not a review, but some comment on a couple doubts i have, hope it
> could be useful...

All comments on the patches and drive-by reviews are always welcome! :-)

> +  if (placeId == -1) // we're in private browsing mode
> +    return NS_OK;
> +
> 
> i'm not really sure about this, placeId == -1 imho should throw and not be
> considered as valid, i fear this could hide other possible bugs or db
> corruptions. Maybe you should directly check if we are in privatebrowsing mode
> through an internal helper method.

You're right.  Done.

> why do you need to create a table to hold only 1 value that is inserted and
> removed sometimes, and you don't need to do searches or manipulation of this
> data? would not be better a pref or another kind of persistent storage (since i
> suppose a user could close the browser in private mode and we should rememeber
> that value to restart in private browsing mode?)?

OK.  A pref would be convenient, but semantically wrong (since the timestamp is not a preference, and that would be abusing the prefs module!), so I decided to go with a temporary file in the profile, to hold a single 64-bit value representing the timestamp.

> caching statements at startup is good for statements executed often, these are
> executed a few times, or never in a session, if you're going to hold the table
> you should probably not cache these since it would regress startup time and not
> provide enough perf gain. And you could directly use executeSimpleSQL for the
> delete.

This is now moot, because I'm no longer storing the timestamp in the DB.
Attachment #342664 - Attachment is obsolete: true
Attachment #342690 - Flags: review?(dietrich)
Attachment #342664 - Flags: review?(dietrich)
Attached patch Patch (v2.12) (obsolete) — Splinter Review
The latest version of the monolithic patch, including the fixes to the individual "for review" patches posted since publishing v2.11.

I've submitted a try server build of this patch which should show up in about 1-2 hours from now.  I'll post a link to the builds page later on.
Attachment #342650 - Attachment is obsolete: true
(In reply to comment #347)
> > why do you need to create a table to hold only 1 value that is inserted and
> > removed sometimes, and you don't need to do searches or manipulation of this
> > data? would not be better a pref or another kind of persistent storage > 
> OK.  A pref would be convenient, but semantically wrong (since the timestamp is
> not a preference, and that would be abusing the prefs module!), so I decided to
> go with a temporary file in the profile, to hold a single 64-bit value
> representing the timestamp.

actually there could be other possibilities rather than creating something new, dietrich could provide better ideas, however i am thinking you could create a "privateBrowsing/lastEntered" itemAnnotation on the places root (nsNavBookmarks->GetPlacesRoot(&placesRoot)) that is a node always valid.
Attachment #342686 - Flags: review?(dwitte) → review+
Comment on attachment 342686 [details] [diff] [review]
[for review] cookie v5

looks good - r=me on the cookieservice bits.
Will it be possible for someone to tell that I have even entered private browsing by looking at the error log or something else?  Of course this is of secondary importance...

And this may be needless work because people use private browsing for different ends... but would it be possible to add a hidden preference in about:config "allow only secure connections in private browsing mode".
(In reply to comment #349)
> actually there could be other possibilities rather than creating something new,
> dietrich could provide better ideas, however i am thinking you could create a
> "privateBrowsing/lastEntered" itemAnnotation on the places root
> (nsNavBookmarks->GetPlacesRoot(&placesRoot)) that is a node always valid.

OK, I'll wait for input from Dietrich (or try to ping him on IRC) to see if there are better/easier possibilities.  I'm willing to change the implementation accordingly.
(In reply to comment #351)
> Will it be possible for someone to tell that I have even entered private
> browsing by looking at the error log or something else?  Of course this is of
> secondary importance...

The current implementation of the places module does create some data on disk which can directly be associated with the time of entering the private browsing mode, so the answer to your question is yes.

> And this may be needless work because people use private browsing for different
> ends... but would it be possible to add a hidden preference in about:config
> "allow only secure connections in private browsing mode".

That should be better off left to a follow-up bug, because it's covering a very specific set of functionality, which might not be appropriate for everyone.  Please file a new bug on this (CC me on it as well) and we can continue the discussion there.
Blocks: 459495
No longer blocks: 459495
Attachment #342690 - Flags: review?(dietrich) → review-
Comment on attachment 342690 [details] [diff] [review]
[for review] places v2


>+static const char* gPrivateBrowsingNotification = "browser:private-browsing";
>+NS_NAMED_LITERAL_STRING(gPBEnter, "enter");
>+NS_NAMED_LITERAL_STRING(gPBLeave, "exit");

nit: would prefer expansion of PB

also, is there a reason these notifications are strings instead of constants defined in the interface? seems like it'd be cleaner if instead of defining these strings all over the codebase, pb-aware code could check against nsIPrivateBrowsing.NOTIFY_ENTER, or some such.

>+  nsCOMPtr<nsIPrivateBrowsingService> pbs =
>+    do_GetService(NS_PRIVATE_BROWSING_SERVICE_CONTRACTID);
>+  if (pbs)
>+    pbs->GetPrivateBrowsing(&mInPrivateBrowsing);

instead of initializing this value in the startup path and checking the property directly, please lazily do this via a smart getter. see bug 342991 for an example.

>+already_AddRefed<nsIFile>
>+nsNavHistory::GetPBEnterDateFile()
>+{
>+  nsCOMPtr<nsIFile> file;
>+  nsresult rv = NS_GetSpecialDirectory(NS_APP_USER_PROFILE_50_DIR,
>+                                       getter_AddRefs(file));
>+  NS_ENSURE_SUCCESS(rv, nsnull);
>+  rv = file->Append(PB_ENTER_FILENAME);
>+  NS_ENSURE_SUCCESS(rv, nsnull);
>+  PRBool exists;
>+  rv = file->Exists(&exists);
>+  NS_ENSURE_SUCCESS(rv, nsnull);
>+  if (exists) {
>+    rv = file->Remove(PR_FALSE);
>+    NS_ENSURE_SUCCESS(rv, nsnull);
>+  }
>+
>+  return file.forget();
>+}

Using a separate file for this is overkill. File i/o can be painfully slow, especially on some Windows systems, USB drives, and mobile devices. Using annotations, as Marco suggested, would incur even more painful file i/o by forcing SQLite to fsync. It may be semantically incorrect, but using a pref here seems to be the most expedient and performant approach. A quick scan of about:config shows that there's a pattern and practice of storing timestamps in preferences.
Blocks: 456884
No longer depends on: 456884
Attached patch Patch (v2.13) (obsolete) — Splinter Review
This patch updates the passwordmgr and places modules as per the review comments, and updates the download manager module to use the latest patch in bug 457110.
Attachment #342691 - Attachment is obsolete: true
So many patches: Could we have a screenshot ... please?
Attached patch [for review] passwordmgr v2 (obsolete) — Splinter Review
(In reply to comment #324)
> (From update of attachment 341415 [details] [diff] [review])
> General nit: patches are easier to review with more context in the diffs... See
> http://developer.mozilla.org/en/Mercurial_FAQ#Configuration (specifically, the
> "-U 8" part).

You're right, sorry about that.

> Like dwitte said, this really seems like it should ask a service for the
> initial state. Something like:
> 
>   var PBS = Cc["mumble"].getService(Ci.mumble);
>   this._inPrivateBrowsing = PBS.inPrivateBrowsing;

Done.

> >+++ b/toolkit/components/passwordmgr/test/subtst_bug_248970_3.html
> ...
> >+function submitForm() {
> >+  setTimeout(function(){ form.submit(); }, 100);
> >+}
> 
> Why the setTimeout()?

That was a left-over from a problem I was haunting in the test and I thought it was related to the timing issues.  I removed it.

> >+++ b/toolkit/components/passwordmgr/test/subtst_bug_248970_4.html
> ...
> >+function submitForm() {
> ...
> >+  var result = pwmgr.autoCompleteSearch("", null, userField);
> >+  if (result instanceof Ci.nsIAutoCompleteResult &&
> >+      result.matchCount == 1) {
> >+    userField.focus();
> >+    userField.value = result.getValueAt(0);
> >+    userField.blur();
> >+  }
> >+
> >+  setTimeout(function(){ form.submit(); }, 100);
> >+}
> 
> Seems like this code should be the top test (test_privbrowsing.html), not in
> the subtest.

Actually I'm not sure if this can be called from the top test without major changes to its logic.  The top test handles the load event on the iframe for subtests by ignore odd loads and handling even loads; and each subtest submits itself, so effectively only the submitted pages are handled by the top test.  The new code in this patch is however cleaner.

> Why call pwmgr.autoCompleteSearch? That's kind of a hacky API, and
> I'd like to remove it eventually... Look at test_basic_form_autocomplete.html
> for how to drive the autocomplete dropdown without this.

OK, done.  I moved doKey to pwmgr_common.js to avoid duplicating it in test_privbrowsing.html.

> >+function get_PBSvc() {
> ...
> >+  if (PBS_CID in Components.classes &&
> >+      "nsIPrivateBrowsingService" in Components.interfaces) {
> >+    try {
> >+      _PBSvc = Components.classes[PBS_CID].
> >+               getService(Components.interfaces.nsIPrivateBrowsingService);
> >+      if (_PBSvc) {
> >+        var observer = {
> ...
> 
> I'm not sure what this is all for. You shouldn't need to check if the service
> Cc/Ci exists (just let it throw+fail if it doesn't),

You're right, done.

> and the observer doesn't seem to do anything useful?

No actually it prevents a dialog from poping up asking you whether to save and close your current session or not.
Attachment #343105 - Flags: review?(dolske)
Attached patch [for review] places v3 (obsolete) — Splinter Review
(In reply to comment #355)
> >+static const char* gPrivateBrowsingNotification = "browser:private-browsing";
> >+NS_NAMED_LITERAL_STRING(gPBEnter, "enter");
> >+NS_NAMED_LITERAL_STRING(gPBLeave, "exit");
> 
> nit: would prefer expansion of PB

Fixed.

> also, is there a reason these notifications are strings instead of constants
> defined in the interface? seems like it'd be cleaner if instead of defining
> these strings all over the codebase, pb-aware code could check against
> nsIPrivateBrowsing.NOTIFY_ENTER, or some such.

They're strings, so I can't define them as const in the IDL, but I went ahead and put preprocessor macros in the IDL and used it in C++ callers.  This should make the code cleaner a bit.

> >+  nsCOMPtr<nsIPrivateBrowsingService> pbs =
> >+    do_GetService(NS_PRIVATE_BROWSING_SERVICE_CONTRACTID);
> >+  if (pbs)
> >+    pbs->GetPrivateBrowsing(&mInPrivateBrowsing);
> 
> instead of initializing this value in the startup path and checking the
> property directly, please lazily do this via a smart getter. see bug 342991 for
> an example.

I think the bug # you cited is incorrect, but I did something which is hopefully what you had in mind in this patch.

> Using a separate file for this is overkill. File i/o can be painfully slow,
> especially on some Windows systems, USB drives, and mobile devices. Using
> annotations, as Marco suggested, would incur even more painful file i/o by
> forcing SQLite to fsync. It may be semantically incorrect, but using a pref
> here seems to be the most expedient and performant approach. A quick scan of
> about:config shows that there's a pattern and practice of storing timestamps in
> preferences.

Removed the whole timestamp-based checks based on our IRC conversation.
Attachment #342690 - Attachment is obsolete: true
Attachment #343106 - Flags: review?(dietrich)
(In reply to comment #357)
> So many patches: Could we have a screenshot ... please?

This is not exactly a bug where a screenshot is helpful, I guess.  But you can download the experimental builds for the latest version of the patch here: <https://build.mozilla.org/tryserver-builds/2008-10-14_12:52-ehsan.akhgari@gmail.com-try-ab9f90709a7/>
Comment on attachment 343106 [details] [diff] [review]
[for review] places v3

> > also, is there a reason these notifications are strings instead of constants
> > defined in the interface? seems like it'd be cleaner if instead of defining
> > these strings all over the codebase, pb-aware code could check against
> > nsIPrivateBrowsing.NOTIFY_ENTER, or some such.
> 
> They're strings, so I can't define them as const in the IDL, but I went ahead
> and put preprocessor macros in the IDL and used it in C++ callers.  This should
> make the code cleaner a bit.

it's still not clear why these are strings, as opposed to integers. while your change helps C++ callers, it doesn't help JS callers at all.

> > >+  nsCOMPtr<nsIPrivateBrowsingService> pbs =
> > >+    do_GetService(NS_PRIVATE_BROWSING_SERVICE_CONTRACTID);
> > >+  if (pbs)
> > >+    pbs->GetPrivateBrowsing(&mInPrivateBrowsing);
> > 
> > instead of initializing this value in the startup path and checking the
> > property directly, please lazily do this via a smart getter. see bug 342991 for
> > an example.
> 
> I think the bug # you cited is incorrect, but I did something which is
> hopefully what you had in mind in this patch.

yes, wrong bug, sorry.

this solution seems over-complicated, shouldn't need 2 properties and 2 methods for it. you should be able to have a private member which holds the value, and a getter which first populates the value if needed, and returns it, right?
(In reply to comment #361)
> it's still not clear why these are strings, as opposed to integers. while your
> change helps C++ callers, it doesn't help JS callers at all.

Because the data parameter to nsIObserver::Observe is a string, and you can't really pass integers to observers this way.  It's true that these changes don't help JS callers, but that's what other interfaces do for their notifications as well (for example, see the offline notifications in the IO service).

> yes, wrong bug, sorry.
> 
> this solution seems over-complicated, shouldn't need 2 properties and 2 methods
> for it. you should be able to have a private member which holds the value, and
> a getter which first populates the value if needed, and returns it, right?

The problem is how to detect if this is the first time we're trying to get the value, so that we can initialize it via the service call.  The only other solution which I could think of besides adding a separate value for tracking this was to store a magical value in mInPrivateBrowsing, which might not be considered good practice as well.

That being said, I don't think that the initialization code is too slow to be called in Init.  It's merely retrieving a value through XPCOM, and it should be acceptably fast.  We're doing something similar for the rest of the modules as well.
>>(In reply to comment #357)
>> So many patches: Could we have a screenshot ... please?
>
>This is not exactly a bug where a screenshot is helpful, I guess.

I'll be posting details about the user interface for private browsing mode to my blog and to bug 411929 hopefully rather soon.
Attached patch [for review] cache v1 (obsolete) — Splinter Review
The cache module patch with a unit test, ready for review.  Darin: please let me know if you're not the right person to ask for review here...  Thanks!
Attachment #343468 - Flags: superreview?(darin.moz)
Attachment #343468 - Flags: review?(darin.moz)
Comment on attachment 343468 [details] [diff] [review]
[for review] cache v1

Switching review to biesi based on IRC conversation with darin.  biesi, please let me know if you're the right person to ask for a review here.
Attachment #343468 - Flags: superreview?(darin.moz)
Attachment #343468 - Flags: superreview?(cbiesinger)
Attachment #343468 - Flags: review?(darin.moz)
Attachment #343468 - Flags: review?(cbiesinger)
Flags: blocking-firefox3.1?
Attached patch [for review] satchel v1 (obsolete) — Splinter Review
The satchel module patch with a unit test, ready for review.
Attachment #343593 - Flags: review?(mconnor)
Attachment #342686 - Flags: review?(gavin.sharp) → review?(mconnor)
Comment on attachment 343106 [details] [diff] [review]
[for review] places v3


>@@ -303,16 +306,18 @@ const PRInt32 nsNavHistory::kAutoComplet
> const PRInt32 nsNavHistory::kAutoCompleteIndex_ParentId = 3;
> const PRInt32 nsNavHistory::kAutoCompleteIndex_BookmarkTitle = 4;
> const PRInt32 nsNavHistory::kAutoCompleteIndex_Tags = 5;
> const PRInt32 nsNavHistory::kAutoCompleteIndex_VisitCount = 6;
> 
> static const char* gQuitApplicationMessage = "quit-application";
> static const char* gXpcomShutdown = "xpcom-shutdown";
> static const char* gAutoCompleteFeedback = "autocomplete-will-enter-text";
>+NS_NAMED_LITERAL_STRING(gPrivateBrowsingEnter, NS_PRIVATE_BROWSING_ENTER);
>+NS_NAMED_LITERAL_STRING(gPrivateBrowsingLeave, NS_PRIVATE_BROWSING_LEAVE);

not necessary to create these globals, just use the constants inline.

(In reply to comment #362)
> > this solution seems over-complicated, shouldn't need 2 properties and 2 methods
> > for it. you should be able to have a private member which holds the value, and
> > a getter which first populates the value if needed, and returns it, right?
> 
> The problem is how to detect if this is the first time we're trying to get the
> value, so that we can initialize it via the service call.  The only other
> solution which I could think of besides adding a separate value for tracking
> this was to store a magical value in mInPrivateBrowsing, which might not be
> considered good practice as well.

i'm ok with initializing mInPrivateBrowsing to a magical value. this would be functional, easily readable, and halves the number of properties and getters used.

> That being said, I don't think that the initialization code is too slow to be
> called in Init.  It's merely retrieving a value through XPCOM, and it should be
> acceptably fast.  We're doing something similar for the rest of the modules as
> well.

i'm reviewing this while in the middle of an epic battle for Ts wins in Places. every nanosecond counts. everything that can reasonably be lazy, should be.
Attachment #343106 - Flags: review?(dietrich) → review-
Attachment #342686 - Flags: superreview?(bzbarsky) → superreview+
Comment on attachment 342686 [details] [diff] [review]
[for review] cookie v5

>+++ b/browser/components/preferences/cookies.js
>@@ -54,40 +54,51 @@ var gCookiesWindow = >+  _populateList: function (aInitiallyLoaded)

Nix the space before '(' unless that's file style?

Also s/aInitiallyLoaded/aInitialLoad/ might be better.

sr=bzbarsky
The nsIPrivateBrowsingService interface, ready for review.
Attachment #343620 - Flags: superreview?(bzbarsky)
Attachment #343620 - Flags: review?(bzbarsky)
(In reply to comment #367)
> not necessary to create these globals, just use the constants inline.

Done.

> i'm ok with initializing mInPrivateBrowsing to a magical value. this would be
> functional, easily readable, and halves the number of properties and getters
> used.

OK, I did that.  I used 0xffffffff as the magical value.

> i'm reviewing this while in the middle of an epic battle for Ts wins in Places.
> every nanosecond counts. everything that can reasonably be lazy, should be.

OK, you definitely know better than me there, so here's the lazy init code  :-)
Attachment #343106 - Attachment is obsolete: true
Attachment #343623 - Flags: review?(dietrich)
bz's nit in comment 368 fixed.  Carrying over dwitte's r+ and bz's sr+.
Attachment #342686 - Attachment is obsolete: true
Attachment #343632 - Flags: review?(mconnor)
Attachment #342686 - Flags: review?(mconnor)
Addressed bz's IRC review comments.
Attachment #343620 - Attachment is obsolete: true
Attachment #343644 - Flags: superreview?(bzbarsky)
Attachment #343644 - Flags: review?(bzbarsky)
Attachment #343620 - Flags: superreview?(bzbarsky)
Attachment #343620 - Flags: review?(bzbarsky)
Changed the implementation of the contentprefs module to not cache the current private browsing status, as per IRC review from mconnor.
Attachment #342661 - Attachment is obsolete: true
Attachment #343647 - Flags: review?(mconnor)
Attachment #342661 - Flags: review?(mconnor)
Comment on attachment 343632 [details] [diff] [review]
[checked in] cookie v6 (Checkin: comment 403)

>+    } else {
>+      if (document.getElementById("filter").value != "")

nit:

}
else {

otherwise r=me on the /browser bits, I assume dwitte and bz got the cookie bits done.
Attachment #343632 - Flags: review?(mconnor) → review+
Comment on attachment 343623 [details] [diff] [review]
[checked in] places v4 (Checkin: comment 404)


>+
>+PRBool
>+nsNavHistory::GetPrivateBrowsingModeInitialState()
>+{
>+  nsCOMPtr<nsIPrivateBrowsingService> pbs =
>+    do_GetService(NS_PRIVATE_BROWSING_SERVICE_CONTRACTID);
>+  if (pbs) {
>+    pbs->GetPrivateBrowsing(&mInPrivateBrowsing);
>+  }
>+
>+  return mInPrivateBrowsing;
>+}

since this is only ever called once and by only one caller, please just move this code into InPrivateBrowsingMode(), and remove this function altogether.

r=me with this fixed.
Attachment #343623 - Flags: review?(dietrich) → review+
Comment on attachment 343647 [details] [diff] [review]
[checked in] contentprefs v4 (Checkin: comment 405)


>     // If the pref is already set to the value, there's nothing more to do.
>     var currentValue = this.getPref(aURI, aName);
>-    if (typeof currentValue != "undefined" && currentValue == aValue)
>-      return;
>+    if (typeof currentValue != "undefined") {
>+      if (currentValue == aValue)
>+        return;
>+    } else {

nit (again):

}
else {

only comment is that it might make sense to make the values used in the test consts, so someone doesn't change one and break the test.  Not really important for unit tests though.

looks good, r=mconnor
Attachment #343647 - Flags: review?(mconnor) → review+
Comment on attachment 343644 [details] [diff] [review]
[checked in] nsIPrivateBrowsingService interface v2 (Checkin: comment 402)

>+++ b/netwerk/base/public/nsIPrivateBrowsingService.idl
>+    // Setting this value while handling one of the notifications generated
>+    // by the private browsing service causes a NS_ERROR_FAILURE exception
>+    // being thrown.

  ... service throws NS_ERROR_FAILURE.

>+ *     set to true to prevent the switch to the private browsing mode.

Is there a good reason for that?  I would have thought that setting to false to mean "no" would make more sense.  Same for leaving.

With those nits, looks great.  I'd be happy to review the corresponding changes to the service impl if that would be helpful.
Attachment #343644 - Flags: superreview?(bzbarsky)
Attachment #343644 - Flags: superreview+
Attachment #343644 - Flags: review?(bzbarsky)
Attachment #343644 - Flags: review+
Comment on attachment 343644 [details] [diff] [review]
[checked in] nsIPrivateBrowsingService interface v2 (Checkin: comment 402)

Hmm, why is the attribute not simply something more descriptive like "enabled" ?

reading consumer code then means you get:

nsIPrivateBrowsingService.enabled instead of nsIPrivateBrowsingService.privateBrowsing

not sure of reasoning, but thought I'd ask
I'd be fine with privateBrowsingEnabled....  The problem with too-generic attributes (or methods) is that if your object implements more than one interface you better hope all identically-named ones are defined to behave the same if it's going to work from JS.  We don't really want to constrain implementation too much by choice of names in interface if we can avoid it.
Comment on attachment 343593 [details] [diff] [review]
[for review] satchel v1

first off, as discussed, we'll kill all of the observer/cached copy stuff and just hit the API directly.  We don't hit this code often enough to justify the overhead.

everything else looks good, so it'll be a quick review when you remove that stuff.
Attachment #343593 - Flags: review?(mconnor)
Attachment #343105 - Flags: review?(dolske) → review+
Comment on attachment 343105 [details] [diff] [review]
[for review] passwordmgr v2

Marking r+ (w00t!), although I'd like to take a look at the updated patch that removes the observers/caching.

>-            var canRememberLogin = this._pwmgr.getLoginSavingEnabled(hostname);
>+            var canRememberLogin = this._inPrivateBrowsing ? false :
>+                                   this._pwmgr.getLoginSavingEnabled(hostname);

Nit: That seems a litle awkward. Better would be the form used elsewhere |foo = !IPB && bar()|. I'd also take appending |if (IPD) CRL = false|.

>+var pb = get_PBSvc();
>+if (!pb) { // Private Browsing might not be available
>+  ok(true, "Private browsing service is not available");
>+  SimpleTest.finish();

I assume other tests (in /browser?) explicitly require the presence of PBS? If some terrible bug should break/disable PBS in Firefox builds, we wouldn't want all the tests passing because they allow for a product to not include it. :)
(In reply to comment #377)
> >+ *     set to true to prevent the switch to the private browsing mode.
> 
> Is there a good reason for that?  I would have thought that setting to false to
> mean "no" would make more sense.  Same for leaving.

Usually such notifications interpret the subject param as the "cancel" param, so if any observers want to bail out, they'd just cancel the request by setting the cancel parameter to true.  I'd be fine either way, but I think we should keep it this way for the sake of consistency.

> With those nits, looks great.  I'd be happy to review the corresponding changes
> to the service impl if that would be helpful.

I'm keeping the service implementation open for changes right now, and I'll need to write a few unit tests for it, but I will post a monolithic patch today, and you're welcome to take a look at the service bits and let me know if you spot anything wrong.  :-)
(In reply to comment #378)
> (From update of attachment 343644 [details] [diff] [review])
> Hmm, why is the attribute not simply something more descriptive like "enabled"
> ?
> 
> reading consumer code then means you get:
> 
> nsIPrivateBrowsingService.enabled instead of
> nsIPrivateBrowsingService.privateBrowsing
> 
> not sure of reasoning, but thought I'd ask

No particular reason, but I wanted to avoid names such as "enabled" for the same reason as Boris mentioned in comment 379.  I think bz's suggestion of privateBrowsingEnabled makes more sense here, so I'll incorporate it in the next iteration (I assume I won't need to ask for another review on the code parts that already have r+ because of only this change in the name of course, right?).
(In reply to comment #375)
> (From update of attachment 343623 [details] [diff] [review])
> since this is only ever called once and by only one caller, please just move
> this code into InPrivateBrowsingMode(), and remove this function altogether.
> 
> r=me with this fixed.

I came up with this code:

  PRBool InPrivateBrowsingMode()
  {
    if (mInPrivateBrowsing == PRIVATEBROWSING_NOTINITED) {
      mInPrivateBrowsing = PR_FALSE;
      nsCOMPtr<nsIPrivateBrowsingService> pbs =
        do_GetService(NS_PRIVATE_BROWSING_SERVICE_CONTRACTID);
      if (pbs) {
        pbs->GetPrivateBrowsing(&mInPrivateBrowsing);
      }
    }

    return mInPrivateBrowsing;
  }

The only difference here is that before trying to get the service, I explicitly set mInPrivateBrowsing to false so that if the service is not available (for example, when places is being used from SeaMonkey or other clients which do not include the private browsing service, the return value becomes correctly false.

Please let me know if you think this change requires another review round, and I'll prepare an updated version of the patch for review.
(In reply to comment #376)
> only comment is that it might make sense to make the values used in the test
> consts, so someone doesn't change one and break the test.  Not really important
> for unit tests though.

Sure, that makes a valid point.  Done.
(In reply to comment #380)
> (From update of attachment 343593 [details] [diff] [review])
> first off, as discussed, we'll kill all of the observer/cached copy stuff and
> just hit the API directly.  We don't hit this code often enough to justify the
> overhead.
> 
> everything else looks good, so it'll be a quick review when you remove that
> stuff.

OK, done.  Here's the new patch up for review.
Attachment #343593 - Attachment is obsolete: true
Attachment #343697 - Flags: review?(mconnor)
Attached patch [for review] passwordmgr v3 (obsolete) — Splinter Review
(In reply to comment #381)
> (From update of attachment 343105 [details] [diff] [review])
> Marking r+ (w00t!), although I'd like to take a look at the updated patch that
> removes the observers/caching.

Sure, here you are! :-)

I moved the _inPrivateBrowsing getter to nsLoginManager, and just had nsLoginManagerPrompter access it through the _pwmgr member.  Also I made a small fix to the test to make the iframe always visible, because the previous approach of making it visible only for the duration of the 9th test might fail in rare cases (since the auto-complete popup requires the iframe to be visible).

> Nit: That seems a litle awkward. Better would be the form used elsewhere |foo =
> !IPB && bar()|. I'd also take appending |if (IPD) CRL = false|.

I went ahead with the latter form.

> I assume other tests (in /browser?) explicitly require the presence of PBS? If
> some terrible bug should break/disable PBS in Firefox builds, we wouldn't want
> all the tests passing because they allow for a product to not include it. :)

Yes, they will!  However, I didn't require presence of the private browsing service in non-browser/ tests keeping other consumers in mind (and also in the hopes of landing this patch piece by piece instead of a single check-in).  But thanks for confirming this!
Attachment #343105 - Attachment is obsolete: true
Attachment #343705 - Flags: review?(dolske)
Attached patch Patch (v2.14) (obsolete) — Splinter Review
The latest and greatest version of the patch based on the recent review comments.  I've also submitted a try server build, and I'll post an update when it's available.
Attachment #342643 - Attachment is obsolete: true
Attachment #343102 - Attachment is obsolete: true
> Usually such notifications interpret the subject param as the "cancel" param,

That's fine with me, but the topic could use a name that makes that a little clearer, then...
(In reply to comment #384)
> 
> Please let me know if you think this change requires another review round, and
> I'll prepare an updated version of the patch for review.

looks fine, r=me
(In reply to comment #389)
> That's fine with me, but the topic could use a name that makes that a little
> clearer, then...

Can you think of a better name?  I followed the pattern of quite-application-requested, offline-requested, and offline-app-requested...
Attachment #343697 - Flags: review?(mconnor) → review+
Comment on attachment 343697 [details] [diff] [review]
[checked in] satchel v2 (Checkin: comment 406)

sweet, looks good! r=mconnor
Ehsan, I've tested your latest try server build and noticed that temporary files which are downloaded to be used with an external application are not deleted when leaving the private browsing mode. They still exist on disk until the browser is closed. But what is more interesting the download manager doesn't store these files in the in-memory database. Instead they end up in the downloads.sqlite.

Should I file a new bug on that? I would prefer this when seeing a comment count of nearly 400 on this bug.
(In reply to comment #394)
> Ehsan, I've tested your latest try server build and noticed that temporary
> files which are downloaded to be used with an external application are not
> deleted when leaving the private browsing mode. They still exist on disk until
> the browser is closed. But what is more interesting the download manager
> doesn't store these files in the in-memory database. Instead they end up in the
> downloads.sqlite.

Oh, that's definitely a bug.  It'd be great if you can file a bug and CC me there.
(In reply to comment #395)
> Oh, that's definitely a bug.  It'd be great if you can file a bug and CC me
> there.

I've filed bug 460608 and bug 460609 for both issues.
No longer depends on: 460609
Keywords: privacy
Should private browsing mode disable geolocation requests? Seems like it should, although I suppose we could just let the user deny the requests (when we prompt them).

Also, I added a line item regarding private browsing to the security review template (https://wiki.mozilla.org/Firefox3/Security_Review_Template), to help make sure that future browser features consider their impact to private browsing.
(In reply to comment #397)
> Should private browsing mode disable geolocation requests? Seems like it
> should, although I suppose we could just let the user deny the requests (when
> we prompt them).

We don't block explicit actions from the user (such as bookmarking), so I'm thinking we shouldn't block geolocation automatically, and should let the user decide instead.  Also, the private browsing mode is really about keeping your browsing history private from other people who use the same computer, more than anything else...

> Also, I added a line item regarding private browsing to the security review
> template (https://wiki.mozilla.org/Firefox3/Security_Review_Template), to help
> make sure that future browser features consider their impact to private
> browsing.

Looks good, thanks!
No longer depends on: 460608, 460609
Depends on: 460608
Depends on: 460609
> Can you think of a better name?  I followed the pattern of
> quite-application-requested, offline-requested, and offline-app-requested

Gah.  And they all use this backwards pattern?  :(  And in our usual charming way, completely undocumented.  Quite lovely.  I hate this undocumented use of the observer service as a backdoor.  :(

Let's try to do better here.  If "true" means cancel, then perhaps the topic should be named "private-browsing-cancellation-opportunity" something?  Might be worth running this by mconnor too, though.  To be honest, I'm having a hard time encoding the "set true to cancel" thing in the name, since that's so backwards to how most things work.  :(
(In reply to comment #399)
> Gah.  And they all use this backwards pattern?  :(  And in our usual charming
> way, completely undocumented.  Quite lovely.  I hate this undocumented use of
> the observer service as a backdoor.  :(

Yes.  I'm wondering if they've passed sr when they were first added to the tree though.

> Let's try to do better here.  If "true" means cancel, then perhaps the topic
> should be named "private-browsing-cancellation-opportunity" something?  Might
> be worth running this by mconnor too, though.  To be honest, I'm having a hard
> time encoding the "set true to cancel" thing in the name, since that's so
> backwards to how most things work.  :(

I'll try to catch mconnor on IRC on this, but I can thought of at least one, less verbose, alternative here: "private-browsing-cancel-vote" (think of the metaphor where an observer "votes" to cancel the private browsing mode change by setting the cancel vote nsISupportsPRBool to true).
> I'm wondering if they've passed sr

Most of them are in modules that don't require sr, as far as I can tell.

"private-browsing-cancel-vote" might work, yeah.
Comment on attachment 343644 [details] [diff] [review]
[checked in] nsIPrivateBrowsingService interface v2 (Checkin: comment 402)

http://hg.mozilla.org/mozilla-central/rev/ab63f0d13ea8
Attachment #343644 - Attachment description: [for review] nsIPrivateBrowsingService interface v2 → [checked in] nsIPrivateBrowsingService interface v2 (Checkin: comment 402)
Comment on attachment 343632 [details] [diff] [review]
[checked in] cookie v6 (Checkin: comment 403)

http://hg.mozilla.org/mozilla-central/rev/53b78b1b3910
Attachment #343632 - Attachment description: [for review] cookie v6 → [checked in] cookie v6 (Checkin: comment 403)
Comment on attachment 343623 [details] [diff] [review]
[checked in] places v4 (Checkin: comment 404)

http://hg.mozilla.org/mozilla-central/rev/e9b859357547
Attachment #343623 - Attachment description: [for review] places v4 → [checked in] places v4 (Checkin: comment 404)
Comment on attachment 343647 [details] [diff] [review]
[checked in] contentprefs v4 (Checkin: comment 405)

http://hg.mozilla.org/mozilla-central/rev/81e6937843a7
Attachment #343647 - Attachment description: [for review] contentprefs v4 → [checked in] contentprefs v4 (Checkin: comment 405)
Comment on attachment 343697 [details] [diff] [review]
[checked in] satchel v2 (Checkin: comment 406)

http://hg.mozilla.org/mozilla-central/rev/ee936d0fd358
Attachment #343697 - Attachment description: [for review] satchel v2 → [checked in] satchel v2 (Checkin: comment 406)
Attached patch Patch (v2.15)Splinter Review
New patch updated to tip after landing the previous five patches.
Attachment #343706 - Attachment is obsolete: true
Comment on attachment 343705 [details] [diff] [review]
[for review] passwordmgr v3

>+    // Whether we are in private browsing mode
>+    get _inPrivateBrowsing() {
>+      // The Private Browsing service might not be available.
>+      try {
>+        var pbs = Cc["@mozilla.org/privatebrowsing;1"].
>+                  getService(Ci.nsIPrivateBrowsingService);
>+        return pbs.privateBrowsingEnabled;
>+      } catch (e) {
>+        return false;
>+      }
>     },

I think it would be better to use a smart getter for the service (with a tweak for when it's not available), so that we don't fetch it repeatedly. EG:

  __privateBrowsingSvc : undefined,
  // If the service isn't available, null will be returned.
  get _privateBrowsingSvc() {
      if (this.__privateBrowsingSvc == undefined) {
          if ("@mozilla.org/privatebrowsing;1" in Cc)
              this.__privateBrowsingSvc = Cc[...].getService(...);
          else
              this.__privateBrowsingSvc = null;
      }
      return this.__privateBrowsingSvc;
  }

and then

  get _inPrivateBrowsing() {
      var svc = this._privateBrowsingService;
      if (svc)
          return svc.privateBrowsingEnabled;
      else
          return false;
  }

+++ b/toolkit/components/passwordmgr/src/nsLoginManagerPrompter.js
...
>-        if (hostname) {
>+        if (hostname && !this._pwmgr._inPrivateBrowsing) {

Err, does this even work?

nsLoginManagerPrompter just obtains _pwmgr as with any other service, so it shouldn't be seeing _inPrivateBrowsing without a corresponding XPCOM interface.

Regardless, because the prompter is a separate component, it shouldn't be using internal pwmgr APIs anyway. Just implement _inPrivateBrowsing locally -- cut'n'paste the original code this patch was putting in nsLoginManger.js (no need to cache the service here, so my last comment doesn't apply).
Attachment #343705 - Flags: review?(dolske) → review-
(In reply to comment #408)
> I think it would be better to use a smart getter for the service (with a tweak
> for when it's not available), so that we don't fetch it repeatedly. EG:

Done.

> +++ b/toolkit/components/passwordmgr/src/nsLoginManagerPrompter.js
> ...
> >-        if (hostname) {
> >+        if (hostname && !this._pwmgr._inPrivateBrowsing) {
> 
> Err, does this even work?
> 
> nsLoginManagerPrompter just obtains _pwmgr as with any other service, so it
> shouldn't be seeing _inPrivateBrowsing without a corresponding XPCOM interface.

Oops, I hadn't noticed that _pwmgr is referring to an XMPCOM component...  I guess the fact that this works is showing some XPCOM wrappers magic, but I agree that it is bad practice.  My bad for not spotting this myself.

> Regardless, because the prompter is a separate component, it shouldn't be using
> internal pwmgr APIs anyway. Just implement _inPrivateBrowsing locally --
> cut'n'paste the original code this patch was putting in nsLoginManger.js (no
> need to cache the service here, so my last comment doesn't apply).

Done.
Attachment #343705 - Attachment is obsolete: true
Attachment #343965 - Flags: review?(dolske)
Attachment #343965 - Flags: review?(dolske) → review+
Comment on attachment 343965 [details] [diff] [review]
[checked in] passwordmgr v4 (Checkin: comment 416)

>+    get _privateBrowsingService() {
>+        if (this.__privateBrowsingService == undefined)
>+            try {
>+                this.__privateBrowsingService = Cc["@mozilla.org/privatebrowsing;1"].
>+                                                getService(Ci.nsIPrivateBrowsingService);
>+            } catch (e) {
>+                this.__privateBrowsingService = null;
>+            }
>+        return this.__privateBrowsingService;
>+    },

One reason to consider using |if ("@mozilla.org/privatebrowsing;1" in Cc)| here (instead of try/catch) is if PBS *is* present but fails to initialize... It's not terribly likely (right now, with the simple init in PBS), but the risk is masking the failure and ignoring private browsing mode for the rest of the session.
(In reply to comment #411)
> One reason to consider using |if ("@mozilla.org/privatebrowsing;1" in Cc)| here
> (instead of try/catch) is if PBS *is* present but fails to initialize... It's
> not terribly likely (right now, with the simple init in PBS), but the risk is
> masking the failure and ignoring private browsing mode for the rest of the
> session.

Sure, I'll do this before landing this piece.
This patch is a small fix to the cookie service code, in order to make it send "reload" notifications on both entering and leaving the private browsing mode.  Previously, it sent a "cleared" notification on entering the private browsing mode, which is semantically wrong (because the cookies weren't actually cleared) and was only needed for the cookie manager window's list to be emptied.  This can be done with "reload".

Misusing the "cleared" notification this way had created a problem with the DOM storage module which uses the "cleared" notification to clear its data, and would potentially hurt add-ons which use this notification as well.
Attachment #343974 - Flags: superreview?(bzbarsky)
Attachment #343974 - Flags: review?(dwitte)
The DOM storage part with a unit test, ready for review.
Attachment #343975 - Flags: superreview?(jonas)
Attachment #343975 - Flags: review?(dcamp)
Comment on attachment 343974 [details] [diff] [review]
[checked in] a small fix to the cookie patch (Checkin: comment 417)

looks good! r=me
Attachment #343974 - Flags: review?(dwitte) → review+
Attachment #343974 - Flags: superreview?(bzbarsky) → superreview+
Comment on attachment 343965 [details] [diff] [review]
[checked in] passwordmgr v4 (Checkin: comment 416)

http://hg.mozilla.org/mozilla-central/rev/79ac062d72c2415
Attachment #343965 - Attachment description: [for review] passwordmgr v4 → [Checked in] passwordmgr v4 (Checkin: comment 415)
Comment on attachment 343974 [details] [diff] [review]
[checked in] a small fix to the cookie patch (Checkin: comment 417)

http://hg.mozilla.org/mozilla-central/rev/65da9468c88b
Attachment #343974 - Attachment description: [for review] a small fix to the cookie patch → [checked in] a small fix to the cookie patch (Checkin: comment 417)
Attachment #343965 - Attachment description: [Checked in] passwordmgr v4 (Checkin: comment 415) → [checked in] passwordmgr v4 (Checkin: comment 416)
(In reply to comment #409)
> Latest try server builds available at:
> <https://build.mozilla.org/tryserver-builds/2008-10-19_15:07-ehsan.akhgari@gmail.com-try-8ddf45ad32c/>

With this try server build it takes about 15 seconds to enter the private browsing mode on OS X. Reverting to normal only takes less a second.
(In reply to comment #418)
> With this try server build it takes about 15 seconds to enter the private
> browsing mode on OS X. Reverting to normal only takes less a second.

Does the same thing happen with <https://build.mozilla.org/tryserver-builds/2008-10-18_05:31-ehsan.akhgari@gmail.com-try-c57e8f00cd7/>?

I don't have a Mac, so I can't test it myself, but can someone else please run the above two versions and report the results?
Comment on attachment 343975 [details] [diff] [review]
[for review] DOM storage v1

So do we really completely want to block storage? Or allow reading but not writing?

What do we do for cookies? Do we allow them to be sent but not written to?
(In reply to comment #420)
> (From update of attachment 343975 [details] [diff] [review])
> So do we really completely want to block storage? Or allow reading but not
> writing?
> 
> What do we do for cookies? Do we allow them to be sent but not written to?

The latter sounds like a better idea, but either way, you probably shouldn't do it in CanUseStorage otherwise a DOM_SECURITY_ERR will be returned. If this propagates to javascript, this will break any sites using DOMStorage while in private browsing.
Returning NS_OK in all the functions with nsDOMStorage::Set* and nsDOMStorageItem::Set* should do the trick, right?

Also, is the code hit often enough that observers are necessary?
I got the impression that that was what other browsers do. Has anybody tested?

Also, what do we do?
Please correct me if I'm wrong.

Looking at the latest cookie patch, we create a new table in memory to store the cookies, so that when private browsing mode is exited we destroy that list and recover the original cookie table as it was before entering private browsing.
(In reply to comment #420)
> (From update of attachment 343975 [details] [diff] [review])
> So do we really completely want to block storage? Or allow reading but not
> writing?

According to the functional spec <https://wiki.mozilla.org/Firefox3.1/PrivateBrowsing/FunctionalSpec#Other>, storage should be blocked entirely.  Of course I'm not sure why mconnor specified it in this way.

> What do we do for cookies? Do we allow them to be sent but not written to?

For cookies, we close the disk DB connection (because the cookie back-end supports using any of the two kinds of storage engines, memory and disk, simultaneously or individually) and use an empty in-memory hash table to store the cookies during the private mode.  When leaving the private mode, that hash table is discarded, and the previous one (as it was when entering the private mode) is restored, and the disk DB connection is re-opened.
(In reply to comment #421)
> The latter sounds like a better idea, but either way, you probably shouldn't do
> it in CanUseStorage otherwise a DOM_SECURITY_ERR will be returned. If this
> propagates to javascript, this will break any sites using DOMStorage while in
> private browsing.

Modifying CanUseStorage was what dcamp suggested in comment 172.  Anyway, site javascript code should be ready to handle this exception because it may be thrown for other reasons as well (for example, if a user has turned off DOM storage via prefs.  Of course whether it's a good idea to throw this exception in this case is another question.

(In reply to comment #422)
> Returning NS_OK in all the functions with nsDOMStorage::Set* and
> nsDOMStorageItem::Set* should do the trick, right?

That's kind of like what we do in other places, such as the contentprefs service, for example.  dcamp: how about changing the implementation according to this plan?

> Also, is the code hit often enough that observers are necessary?

It's not called enough, but I'm ready to turn off caching the preference value if the reviewers request it (we've done this in a number of other modules recently).
(In reply to comment #421)
> The latter sounds like a better idea, but either way, you probably shouldn't do
> it in CanUseStorage otherwise a DOM_SECURITY_ERR will be returned. If this
> propagates to javascript, this will break any sites using DOMStorage while in
> private browsing.

I'm not sure it's better to silently fail to do what they asked.  A well-written site needs to be ready for storage exceptions anyway (they can be thrown for quota reasons, for example).
So from a users point of view all cookies will 'disappear' while in private browsing mode? And then come back when private browsing ends? (disregarding what appears in the prefs UI)
(In reply to comment #428)
> So from a users point of view all cookies will 'disappear' while in private
> browsing mode? And then come back when private browsing ends? (disregarding
> what appears in the prefs UI)

yes.
(In reply to comment #428)
> So from a users point of view all cookies will 'disappear' while in private
> browsing mode? And then come back when private browsing ends? (disregarding
> what appears in the prefs UI)

The prefs UI will remain updated as well, so if someone leaves the cookies manager dialog open, she will see this happening.
Comment on attachment 343468 [details] [diff] [review]
[for review] cache v1

+++ b/netwerk/cache/src/nsCacheService.cpp

in Observe():

-            nsCacheService::SetDiskCacheEnabled(DiskCacheEnabled());
+            if (mInPrivateBrowsing)
+                mDiskCacheEnabled = PR_FALSE;

Hm... why set mDiskCacheEnabled to false here? It should be false already when mInPrivateBrowsing is true, right?

-            nsCacheService::SetOfflineCacheEnabled(OfflineCacheEnabled());
+            if (mInPrivateBrowsing)
+                mOfflineCacheEnabled = PR_FALSE;

same here.

+    } else if (!strcmp(NS_PRIVATE_BROWSING_SWITCH_TOPIC, topic)) {
+
+        nsCOMPtr<nsIPrefBranch> branch = do_GetService(NS_PREFSERVICE_CONTRACTID);

please remove the empty line

also, you only need the prefbranch in the else-branch of the if, right? Can you move it there?

+            rv = branch->GetBoolPref(DISK_CACHE_ENABLE_PREF,
+                                     &mDiskCacheEnabled);
+            if (NS_FAILED(rv))  return rv;
+            nsCacheService::SetDiskCacheEnabled(DiskCacheEnabled());

You shouldn't treat this as a fatal failure. Like ReadPrefs(), you should instead default to true if getting the pref fails.

Same for the offline cache pref.

In ReadPrefs():
+    if (mInPrivateBrowsing)
+        mDiskCacheEnabled = PR_FALSE;

again, this explicit setting shouldn't be needed.

+nsCacheService::OnEnterExitPrivateBrowsing()

Why do you clear the memory cache and doom the active entries when entering the private browsing mode?

I guess that may be an answer for the spec author instead...

+++ b/netwerk/test/unit/test_bug248970_cache.js
+    cache_prefs[pref] = prefs.getBoolPref(pref);

there's not really a need for that. they all default to true.

+function get_CacheService() {

why the strange naming? Why not get_cache_service(), consistent with all the other functions? similar for get_PBSvc.

+        return dirSvc.get("CurProcD", Ci.nsILocalFile);

That doesn't seem like the best possible choice, why not use something below TmpD?

+  var visitor = {
+    QueryInterface: function (iid) {

no point in writing this QueryInterface function, xpconnect will synthesize it for you. (also in the other places where you have a visitor)

+  // see if any of the required devices was missing
+  for (var i = 0; i < devices.length; ++ i) {

Wouldn't it be much simpler to check devices.sort().toString() != found_devices.sort().toString()?

+function make_input_stream_scriptable(input)
+{
+    var wrapper = Cc["@mozilla.org/scriptableinputstream;1"].

use consistent formatting. the other functions put the { on the same line as the function declaration and use two-space indentation.

+    var cs = get_CacheService();
+    do_check_neq(cs, null);

that function will never return null. it will either throw or return a valid service.

+    store_in_cache("cache-A", "test content", "memory");

Instead of repeating the "memory", "disk", "offline" strings all over the place, can you define constants for them?

also, can you define a constant for "test content" instead of repeating it for the writing and reading functions?
Attachment #343468 - Flags: superreview?(cbiesinger)
Attachment #343468 - Flags: review?(cbiesinger)
Attachment #343468 - Flags: review-
(In reply to comment #419)
> > With this try server build it takes about 15 seconds to enter the private
> > browsing mode on OS X. Reverting to normal only takes less a second.
> 
> I don't have a Mac, so I can't test it myself, but can someone else please run
> the above two versions and report the results?

Yes, it also occurs when using that build. But at least its a profile issue. I'll figure out what happens and file a new bug.
I was not able to reproduce what Henrik reports in Comment 419, using an existing profile, but as he notes in Comment 432 it seems to be a profile issue.
Attached patch [for review] cache v2 (obsolete) — Splinter Review
(In reply to comment #431)
> +nsCacheService::OnEnterExitPrivateBrowsing()
> 
> Why do you clear the memory cache and doom the active entries when entering the
> private browsing mode?
> 
> I guess that may be an answer for the spec author instead...

Yes, I just followed the functional spec <https://wiki.mozilla.org/Firefox3.1/PrivateBrowsing/FunctionalSpec#Cache>.  The reasoning behind that decision is not very clear to me, because what we're trying to do is prevent anything from leaking *from* the private mode to the non-private mode, not vice versa, but anyway if mconnor has a different idea in this regard, I'm all for it.

All other comments were fixed in the new version of the patch.
Attachment #343468 - Attachment is obsolete: true
Attachment #344340 - Flags: superreview?(cbiesinger)
Attachment #344340 - Flags: review?(cbiesinger)
I think the functional spec there is mistaken; this bug and this initial work is and should be about eliminating _local_ traces.  Preventing sites from tracking you seems out of scope, especially without a clearer description of the adversary (remote site? ISP? safebrowsing provider?).

I've worked on pseudonymity and profile-splitting software before, it's really hard to get right, especially in light of various (deployed) deep packet inspection tools.  We would do well to not do a half job of it, and mis-set user expectation; rather we should implement a solid version of "leave no local trace",
Attachment #344340 - Flags: superreview?(cbiesinger)
Attachment #344340 - Flags: superreview+
Attachment #344340 - Flags: review?(dcamp)
Attachment #344340 - Flags: review?(cbiesinger)
Attachment #344340 - Flags: review+
Comment on attachment 344340 [details] [diff] [review]
[for review] cache v2

Dave: Do you know the cache service well enough to tell whether the ClearDoomList() call in OnEnterExitPrivateBrowsing() is necessary/a good thing?

Ehsan:
    do_throw("Expected to find the \"" + devices[i] + "\" cache device, " +

now that you have no for loop anymore, you don't have "i" anymore :-)
(In reply to comment #436)
> (From update of attachment 344340 [details] [diff] [review])
> Dave: Do you know the cache service well enough to tell whether the
> ClearDoomList() call in OnEnterExitPrivateBrowsing() is necessary/a good thing?

As I understand it, ClearDoomList() will deactivate all cache entries currently in flight.  That'll cause problems with channels loading into/from those cache entries.

It should be fine to just doom everything.  As loads finish, the doom list will be cleaned up.
Blocks: 461221
Comment on attachment 343848 [details] [diff] [review]
Patch (v2.15)

Asking mconnor for a primitive review on the browser/ bits and pieces.
Attachment #343848 - Flags: review?(mconnor)
Here's the revised patch, without calling ClearDoomsList, and with the problem mentioned in comment 436 fixed.
Attachment #344340 - Attachment is obsolete: true
Attachment #344444 - Flags: review?(dcamp)
Attachment #344340 - Flags: review?(dcamp)
(In reply to comment #435)
> I think the functional spec there is mistaken; this bug and this initial work
> is and should be about eliminating _local_ traces.  Preventing sites from
> tracking you seems out of scope, especially without a clearer description of
> the adversary (remote site? ISP? safebrowsing provider?).
> 
> I've worked on pseudonymity and profile-splitting software before, it's really
> hard to get right, especially in light of various (deployed) deep packet
> inspection tools.  We would do well to not do a half job of it, and mis-set
> user expectation; rather we should implement a solid version of "leave no
> local trace",

So if we're just trying to prevent local traces then we should *not* do things like remove all currently set cookies, right? As well as not deny access to *reading* localStorage.

My understanding is that this is what other UAs do. I.e. they leave all existing cookies and history, and just don't permanently add any additional ones.

If this is what we want to do too then I think the spec needs to be changed quite a bit.
(In reply to comment #440)
> My understanding is that this is what other UAs do. I.e. they leave all
> existing cookies and history, and just don't permanently add any additional
> ones.

I don't think that's what Chrome is doing. If you enter private mode and open google.com, you won't be logged in with your google account (meaning the existing cookies are not sent I guess).
(In reply to comment #440)
> So if we're just trying to prevent local traces then we should *not* do things
> like remove all currently set cookies, right? As well as not deny access to
> *reading* localStorage.

Based on our IRC discussions with mconnor last night, he would like to prevent some types of attacks which enable a website to associate your non-private and private browsing sessions together.  We won't of course make any promises in this regard to users, because we're not covering every possible ground here.  The reason that the spec says that reading DOM storage should be blocked as well is to prevent this association.

> My understanding is that this is what other UAs do. I.e. they leave all
> existing cookies and history, and just don't permanently add any additional
> ones.

Well, I tested Chrome, and like Sylvian mentioned in comment 441, it indeed doesn't send non-private cookies in private mode.  It seems that Chrome doesn't support DOM storage though, but I suppose if it did, it would've made the same choice as for the cookies.  Both my patch and Chrome's implementation keep history in place.  The only difference in our handling of history is that we turn off visited link coloring in private mode to prevent attacks like <http://www.azarask.in/blog/post/socialhistoryjs/>.

> If this is what we want to do too then I think the spec needs to be changed
> quite a bit.

It seems that for the time being, we're going to maintain the current spec, and we need to make sure that the implementation follows the spec exactly.
Attachment #344444 - Flags: review?(dcamp) → review+
Attached patch [for review] sessionstore v1 (obsolete) — Splinter Review
The session store module changes with tests, ready for review.
Attachment #344483 - Flags: review?(dietrich)
Attachment #344483 - Flags: review?(dietrich)
Comment on attachment 344483 [details] [diff] [review]
[for review] sessionstore v1

>+    // The Private Browsing service might not be available.
>+    try {
>+      var pbs = Cc["@mozilla.org/privatebrowsing;1"].
>+                getService(Ci.nsIPrivateBrowsingService);
>+      this._inPrivateBrowsing = pbs.privateBrowsingEnabled;
>+    } catch (e) {}

Isn't the PB service part of /browser as is SessionStore? If so, please drop the try-catch clause.

>+   * @param aBackupState
>+   *        Bool backup the current state

Why can't you just do

>     this._stateBackup = this._safeEval(this._getCurrentState(true).toSource());

while you handle PB's "enter" notification? IMO saveState shouldn't mean saveStateOrMaybeJustBackupWithoutSaving. You don't have to worry about the state's session object at this point (and later on, only session.state is needed).

>+      this._stateBackup = eval(oState.toSource());

Our toSource implementation still has several quirks (which I suspect cause some of the dependencies of bug 429414). Please use _safeEval instead.

>+  _saveBackedUpState: function sss_saveBackedUpState() {
>+    // if crash recovery is disabled, only save session resuming information
>+    if (!this._resume_from_crash && this._loadState == STATE_RUNNING)
>+      return;
>+    if (!("_stateBackup" in this))
>+      return;

Do we ever hit this method under these circumstances? If not, I'd rather just move the remaining three lines to where they're actually needed.

>+  _backupRecentlyClosedTabs: function sss_backupRecentlyClosedTabs() {

I'm still opposed to this. ;-)

>+      backup[ix] = eval(this._windows[ix]._closedTabs.toSource());

_safeEval!

As for the tests, I've unfortunately found some nits as well (many apply to both tests):

>+var _PBSvc = null;
>+function get_PBSvc() {
>+ if (_PBSvc)
>+  return _PBSvc;

Smart getter, please.

>+   if (_PBSvc) {

This should always happen - unless Firefox can be built without PB service (in which case I must've missed a lot of ifdefs). Otherwise you can quite simplify these bits.

>+       QueryInterface: function (iid) {

This isn't needed for JS components which don't implement nsISupportsWeakReference.

>+function test() {  
>+  /** Test (A) for Bug 248970 **/
>+
>+  function test(aLambda) {
>+   try {
>+     return aLambda() || true;
>+   }
>+   catch(ex){ }
>+     return false;
>+  }

While this is just a test, I'd still prefer trying to stick to our coding standards: no whitespace at the ends of lines; proper identation; spaces after catch and (ex).

>+  let tabbrowser = getBrowser();

Just use gBrowser to start with, getBrowser has been deprecated.

>+  ok(pb, "reference the private browsing service");

This should be in a PB specific test - here it should just work!

>+  ok(ssComponent, "reference the sessionstore component");
>+  ok(ss, "reference the sessionstore service");

We already test for this - here it should just work as well!

>+    ok(file,"reference the profile directory");

I'd prefer you only testing relevant functionality - makes the test's output easier to skim.

>+      let ss_lmt_pre_pb = file.lastModifiedTime;

What about prePBModeTime or something slightly more readable?

>+          finish();
>+          }, 3000);
>+          }, 3000);

Indentation?

>+      let tab_A_restored = test(function() ss.undoCloseTab(window, 0));
>+      ok(tab_A_restored, "a tab is in undo list");

ss.undoCloseTab will throw when the tab isn't in the undo list!

The tests look good, though. Thanks for your efforts!
Attached patch [for review] sessionstore v2 (obsolete) — Splinter Review
(In reply to comment #444)
> (From update of attachment 344483 [details] [diff] [review])
> >+    // The Private Browsing service might not be available.
> >+    try {
> >+      var pbs = Cc["@mozilla.org/privatebrowsing;1"].
> >+                getService(Ci.nsIPrivateBrowsingService);
> >+      this._inPrivateBrowsing = pbs.privateBrowsingEnabled;
> >+    } catch (e) {}
> 
> Isn't the PB service part of /browser as is SessionStore? If so, please drop
> the try-catch clause.

Yes, fixed.

> >+   * @param aBackupState
> >+   *        Bool backup the current state
> 
> Why can't you just do
> 
> >     this._stateBackup = this._safeEval(this._getCurrentState(true).toSource());
> 
> while you handle PB's "enter" notification? IMO saveState shouldn't mean
> saveStateOrMaybeJustBackupWithoutSaving.

You're right, I don't know what I was thinking!  :-)  Done.

> You don't have to worry about the
> state's session object at this point (and later on, only session.state is
> needed).

I'm not sure what you mean here.  I don't touch the session object here...

> >+      this._stateBackup = eval(oState.toSource());
> 
> Our toSource implementation still has several quirks (which I suspect cause
> some of the dependencies of bug 429414). Please use _safeEval instead.

Done.

> >+  _saveBackedUpState: function sss_saveBackedUpState() {
> >+    // if crash recovery is disabled, only save session resuming information
> >+    if (!this._resume_from_crash && this._loadState == STATE_RUNNING)
> >+      return;
> >+    if (!("_stateBackup" in this))
> >+      return;
> 
> Do we ever hit this method under these circumstances? If not, I'd rather just
> move the remaining three lines to where they're actually needed.

I don't think the first condition matters, now that I think about it.  There is a potential for the second condition.  For example, the implementation of bug 460346 might cause situations where the enter notification is not received before exit.  Therefore I prefer to keep it, at least as a sanity check, as the case may be.

> >+  _backupRecentlyClosedTabs: function sss_backupRecentlyClosedTabs() {
> 
> I'm still opposed to this. ;-)

OK, I've given this extensive thought.  With this implementation, if the user doesn't choose to save and close her current session, after leaving the private mode, her browser is exactly at the same state as when entering the private mode.  The same thing happens if she chooses to save and close her session (even without this code).  So, this provides a level of consistency between the two options.

Another point here is that I've gone to great lengths to make it impossible for a someone to tell whether the private mode has even been entered by inspecting two states of the browser (which might be before and after the private session).  Without this code, this will break.

Thirdly, since in the final implementation that we will ship, the option to save and close the current session might be the only available option facing the users, this might not be that debatable.

So, unless there are technical reasons that we should remove this code, or I'm doing something wrong in it, I prefer to keep it.

> >+      backup[ix] = eval(this._windows[ix]._closedTabs.toSource());
> 
> _safeEval!

Done.

> As for the tests, I've unfortunately found some nits as well (many apply to
> both tests):
> 
> >+var _PBSvc = null;
> >+function get_PBSvc() {
> >+ if (_PBSvc)
> >+  return _PBSvc;
> 
> Smart getter, please.

Hum?  Are getters possible at global scope?

> >+   if (_PBSvc) {
> 
> This should always happen - unless Firefox can be built without PB service (in
> which case I must've missed a lot of ifdefs). Otherwise you can quite simplify
> these bits.

No you didn't miss them.  My bad for directly pasting this code from unit tests outside of browser/!  :-)

> >+       QueryInterface: function (iid) {
> 
> This isn't needed for JS components which don't implement
> nsISupportsWeakReference.

Oh, cool, I didn't know that!

> >+function test() {  
> >+  /** Test (A) for Bug 248970 **/
> >+
> >+  function test(aLambda) {
> >+   try {
> >+     return aLambda() || true;
> >+   }
> >+   catch(ex){ }
> >+     return false;
> >+  }
> 
> While this is just a test, I'd still prefer trying to stick to our coding
> standards: no whitespace at the ends of lines; proper identation; spaces after
> catch and (ex).

Done.

> >+  let tabbrowser = getBrowser();
> 
> Just use gBrowser to start with, getBrowser has been deprecated.

Sure.

> >+  ok(pb, "reference the private browsing service");
> 
> This should be in a PB specific test - here it should just work!

Agreed.

> >+  ok(ssComponent, "reference the sessionstore component");
> >+  ok(ss, "reference the sessionstore service");
> 
> We already test for this - here it should just work as well!

Done.

> >+    ok(file,"reference the profile directory");
> 
> I'd prefer you only testing relevant functionality - makes the test's output
> easier to skim.

Definitely.

> >+      let ss_lmt_pre_pb = file.lastModifiedTime;
> 
> What about prePBModeTime or something slightly more readable?

prePBModeTimeStamp (slightly more readable)!  :-)

> >+          finish();
> >+          }, 3000);
> >+          }, 3000);
> 
> Indentation?

What about it?  ;-)

> >+      let tab_A_restored = test(function() ss.undoCloseTab(window, 0));
> >+      ok(tab_A_restored, "a tab is in undo list");
> 
> ss.undoCloseTab will throw when the tab isn't in the undo list!

In this case, |test| should return false, right?

> The tests look good, though. Thanks for your efforts!

I can't take credit for them (but I *am* tempted to)!  The tests are Aaron Train's great work.

I hope you don't mind me shamelessly switching the review flag to you...
Attachment #344483 - Attachment is obsolete: true
Attachment #344558 - Flags: review?(zeniko)
Thank you, Ehsan and Simon. With that testing area complete, what now remains?
(In reply to comment #446)
> Thank you, Ehsan and Simon. With that testing area complete, what now remains?

Good question.  Here's a status update.  The only tests remaining are those of the download manager (which I need to write a test plan for) and the private browsing service itself.  I expect to complete them in the next couple of days (at most).  One we have the reviews on the remaining work, it will be ready to land!
Attachment #344558 - Flags: review?(zeniko) → review+
Comment on attachment 344558 [details] [diff] [review]
[for review] sessionstore v2

r+=me with the following changes:

>+            oState.session.state = STATE_STOPPED_STR;

This doesn't work, as the session object is only set in saveState right before writing to a file. Instead do

>>            oState.session = { state: STATE_STOPPED_STR };

>+          this._inPrivateBrowsing = false;

Please also delete this._stateBackup at this point (or better just always delete it at the end of the "exit" block).

>+function get_PBSvc() {

On second thought: Just inline this in the main test body. There's no need for caching if you only ever need the service once. See the browser_448741.js test.

>+  let ssComponent = test(function() Cc["@mozilla.org/browser/sessionstore;1"]);
>+  let ss = test(function() ssComponent.getService(Ci.nsISessionStore));
>+  if (pb) { // private browsing might not be available

You can just get the service in one swoop. And |pb| will always be the PB service, so you can just drop the |if|.

(In reply to comment #445)
> Another point here is that I've gone to great lengths to make it impossible for
> a someone to tell whether the private mode has even been entered by inspecting
> two states of the browser (which might be before and after the private
> session).  Without this code, this will break.

Good point! No further objections.

> Hum?  Are getters possible at global scope?

They are (see e.g. Dão's clean-up in browser.js). Doesn't matter with the above changes, though.

> In this case, |test| should return false, right?

Of course - I completely missed the |test|.

> The tests are Aaron Train's great work.

In that case thanks to Aaron as well!
Comment on attachment 344558 [details] [diff] [review]
[for review] sessionstore v2

One additional detail about the first test: That one might actually pass even though sessionstore.js would have been overwritten. You'll want to set browser.sessionstore.interval to 1000 at the start of the test (and reset the value at the end) to make sure that the file would indeed have been updated during the course of the test...
So microsoft does not drop any existing cookies when going in to "InPrivate mode". They were simply trying to address the problem of local traces.

Apparently apple do the same thing in safari.

I don't really have a strong opinion on what is the best thing to do, though I'm wondering what problem we're trying to address by dropping cookies are.
Attached patch [for review] sessionstore v3 (obsolete) — Splinter Review
(In reply to comment #448)
> (From update of attachment 344558 [details] [diff] [review])
> r+=me with the following changes:
> 
> >+            oState.session.state = STATE_STOPPED_STR;
> 
> This doesn't work, as the session object is only set in saveState right before
> writing to a file. Instead do
> 
> >>            oState.session = { state: STATE_STOPPED_STR };

I did this, but I see a potential problem with this approach (not having saveState save the backup state object):  In this case, the session object excludes the lastUpdate, recentCrashes, and any other attributes saved by saveState later on.  Wouldn't this be a problem?  I think this was the original reason why I was doing the state backup saving in saveState initially.

Besides what I was doing before, another (better) solution might be to factor out the code preparing the state object for being written to disk into a function, such as _prepareState and then call it both from saveState and the private-browsing observer.

> >+          this._inPrivateBrowsing = false;
> 
> Please also delete this._stateBackup at this point (or better just always
> delete it at the end of the "exit" block).

Done.

> >+function get_PBSvc() {
> 
> On second thought: Just inline this in the main test body. There's no need for
> caching if you only ever need the service once. See the browser_448741.js test.

Done.

> >+  let ssComponent = test(function() Cc["@mozilla.org/browser/sessionstore;1"]);
> >+  let ss = test(function() ssComponent.getService(Ci.nsISessionStore));
> >+  if (pb) { // private browsing might not be available
> 
> You can just get the service in one swoop. And |pb| will always be the PB
> service, so you can just drop the |if|.

You're right, didn't catch this before.

(In reply to comment #449)
> (From update of attachment 344558 [details] [diff] [review])
> One additional detail about the first test: That one might actually pass even
> though sessionstore.js would have been overwritten. You'll want to set
> browser.sessionstore.interval to 1000 at the start of the test (and reset the
> value at the end) to make sure that the file would indeed have been updated
> during the course of the test...

Good point, done.

Carrying forward your r+, however, another iteration might be necessary because of the session object point I mentioned above.
Attachment #344558 - Attachment is obsolete: true
Attachment #344729 - Flags: review+
Depends on: 461625
Depends on: 461627
(In reply to comment #450)
> I don't really have a strong opinion on what is the best thing to do, though
> I'm wondering what problem we're trying to address by dropping cookies are.

Well, the reasoning behind this is that one of the applications of cookies is
tracking the visitors of a website.  For example, if a site sets a cookie with
a unique ID the first time you visit it (like Google does) and your browser
sends that cookie for each request, the website effectively has a log of all
your activities associated together.

Now, if your browser send that cookie inside the private mode, the site would
be able to associate your public and private activities.  An example of how
this manifests to a user is that the Google searches done in private mode show
up in her search history.  This might be a problem for shared computers, and
it's what we're trying to avoid by not sending any of the existing cookies
inside the private mode.
(In reply to comment #451)
> In this case, the session object excludes the lastUpdate, recentCrashes,
> and any other attributes saved by saveState later on.  Wouldn't this be a
> problem?

lastUpdate and recentCrashes are both not needed after a clean shut-down. We can revisit this once we add further information to .session which will always have to be available (in which case we might as well add the .session object in _getCurrentState already).

One further thing that missed during my review: You'll have to remove the observers during a test's clean-up so that they don't have unintended side-effects on later tests. Sorry for that.
Depends on: 461629
Attached patch [for review] sessionstore v4 (obsolete) — Splinter Review
(In reply to comment #453)
> lastUpdate and recentCrashes are both not needed after a clean shut-down. We
> can revisit this once we add further information to .session which will always
> have to be available (in which case we might as well add the .session object in
> _getCurrentState already).

OK.

> One further thing that missed during my review: You'll have to remove the
> observers during a test's clean-up so that they don't have unintended
> side-effects on later tests. Sorry for that.

Oh, the mistake was on my part.  Actually I think we need to do this for the passwordmgr test as well.  I've filed bug 461629 on that.
Attachment #344729 - Attachment is obsolete: true
Attachment #344732 - Flags: review?(zeniko)
Attachment #344444 - Attachment description: [for review] cache v3 → [checked in] cache v3 (Checkin: comment 455)
Comment on attachment 344732 [details] [diff] [review]
[for review] sessionstore v4

Thanks. BTW: Are you aware that your patches aren't UTF-8 encoded and use DOS line endings? The former prevents it from cleanly applying - at least when using "patch" (and the latter leads to console noise). This shouldn't be a problem if you push the patches yourself, though.
Attachment #344732 - Flags: review?(zeniko) → review+
(In reply to comment #456)
> (From update of attachment 344732 [details] [diff] [review])
> Thanks. BTW: Are you aware that your patches aren't UTF-8 encoded and use DOS
> line endings? The former prevents it from cleanly applying - at least when
> using "patch" (and the latter leads to console noise). This shouldn't be a
> problem if you push the patches yourself, though.

Thanks for pointing this out.  I manage my patches using mq, but I generate the splitted patches manually using Notepad++, and that was an oversight on my part.  Sorry for the inconvenience.
Attached patch [for review] service & UI v1 (obsolete) — Splinter Review
This patch includes both the Private Browsing service, and its temporary UI (that is, until bug 411929 is planned and fixed.

I have written some extensive unit tests to check nearly every aspect of the private browsing service.  I have also written a browser chrome test to test the private browsing UI pieces.

This is ready for review now.
Attachment #344819 - Flags: review?(mconnor)
Attachment #343848 - Flags: review?(mconnor)
Depends on: 461710
Depends on: 461714
Latest try server build available at: <https://build.mozilla.org/tryserver-builds/2008-10-26_12:14-ehsan.akhgari@gmail.com-try-535d9d8935f/>

This build includes the patches to bug 461625 and bug 461627 as well (well, but 461710 too, but that one's just a unit test, without any user-facing features).
Latest try server build available at: <https://build.mozilla.org/tryserver-builds/2008-10-26_16:54-ehsan.akhgari@gmail.com-try-2ddbcd278de/>

This build includes the patch to bug 460346 as well (adds the browser.privatebrowsing.autostart pref).
Blocks: 461747
Blocks: 461748
Blocks: 461749
Blocks: 461750
Depends on: 461755
Another random thought: should private browsing mode clear the clipboard when
exiting, if the contents were copied while in private mode? EG, user enters PB,
copies a link on cheap-engagement-rings.com, then exits PB mode. That link
remains in the clipboard, where someone might discover it if they paste it into
something later on.
(In reply to comment #461)
> Another random thought: should private browsing mode clear the clipboard when
> exiting, if the contents were copied while in private mode? EG, user enters PB,
> copies a link on cheap-engagement-rings.com, then exits PB mode. That link
> remains in the clipboard, where someone might discover it if they paste it into
> something later on.

Yeah, I think that would also be a nice thing to handle.  Can you please file a bug on that, and CC me/assign it to me?  We can have a discussion there.
No longer blocks: 461747, 461748, 461749, 461750
I'd prefer to not land temporary UI in locales/en-US, that's just distracting our localization community. I'd recommend to land the localizable files in content and move them over once the UI finalizes.

Style comment, please add localization notes for replaced variables like message in privatebrowsing.properties, https://developer.mozilla.org/En/Localization_notes on the impl.
(In reply to comment #463)
> I'd prefer to not land temporary UI in locales/en-US, that's just distracting
> our localization community. I'd recommend to land the localizable files in
> content and move them over once the UI finalizes.
> 
> Style comment, please add localization notes for replaced variables like
> message in privatebrowsing.properties,
> https://developer.mozilla.org/En/Localization_notes on the impl.

I think the browser/ strings and the parts of the UI code which use them can land in the UI bug with the final strings.  The current stuff is merely a test UI to make the process of testing the builds by the community easier.

There are strings for the download manager as well, but I'm not sure if we need to move them to the UI bug as well.  Alex?
(In reply to comment #461)
> Another random thought: should private browsing mode clear the clipboard when
> exiting, if the contents were copied while in private mode? EG, user enters PB,
> copies a link on cheap-engagement-rings.com, then exits PB mode. That link
> remains in the clipboard, where someone might discover it if they paste it into
> something later on.

I must dissagree. I think copying something to the clipboard is "user data" ie. something the user explicitly did and we shouldn't be touching it.

An option.. sure. But completely clearing the clipboard is a really irritating thing that's hard to track down.
Presumably clipboard clearing would happen *only* when (1) a cut/copy was performed in PB mode and (2) when exiting  PB the clipboard is still that value. A clipboard value that either predates entering PB mode or comes from another app while in PB mode shouldn't be cleared. That would indeed be annoying.
Depends on: 462106
Bug 462106 filed for discussing what to do with the clipboard data.
Comment on attachment 344819 [details] [diff] [review]
[for review] service & UI v1

>diff --git a/browser/base/content/browser.js b/browser/base/content/browser.js

>+  init: function PBUI_init() {
>+    let Cc = Components.classes; // XXXehsan: why can't we use the global Cc here???

as discussed on IRC, this is because Cc isn't defined when you call this.  We really need to clean up Cc, its this weird sorta-const that isn't the same as Ci and Cu.

>+    } else if (aTopic == "quit-application") {

don't you have enough nits on this yet?

>+gPrivateBrowsingUI.init();

why is this called this early?  there's no need to call it there...

>diff --git a/browser/components/nsBrowserGlue.js b/browser/components/nsBrowserGlue.js

>+  decidePrivateBrowsingSession: function()
>+  {

>+    return result;
>+  },

we don't want this anymore.  Its a whole bunch of code that enables a hidden pref.  have the pref for unit tests, kill the UI/prompt.


>+  // Whether the private browsing mode is currently active or not.
>+  _inPrivateBrowsing: false,
>+
>+  // Saved browser state before entering the private mode.
>+  _savedBrowserState: null,
>+
>+  // Whether we're in the process of shutting down
>+  _quitting: false,
>+
>+  // How to treat the non-private session
>+  _sessionDecision: -1,
>+
>+  // Make sure we don't allow re-enterant changing of the private mode
>+  _alreadyChangingMode: false,
>+
>+  // XPCOM registration
>+  classDescription: "PrivateBrowsing Service",
>+  contractID: "@mozilla.org/privatebrowsing;1",
>+  classID: Components.ID("{c31f4883-839b-45f6-82ad-a6a9bc5ad599}"),
>+  _xpcom_categories: [
>+    { category: "app-startup", service: true }
>+  ],
>+
>+  QueryInterface: XPCOMUtils.generateQI([Ci.nsIPrivateBrowsingService, 
>+                                         Ci.nsIObserver]),
>+
>+  _unload: function PBS__destroy() {
>+    // Force an exit from the private browsing mode on shutdown
>+    this._quitting = true;
>+    if (this._inPrivateBrowsing)
>+      this.privateBrowsingEnabled = false;
>+
>+    this._obs.removeObserver(this, "quit-application-granted");
>+  },
>+
>+  _onPrivateBrowsingModeChanged: function PBS__onPrivateBrowsingModeChanged() {
>+    // clear all auth tokens
>+    let sdr = Cc["@mozilla.org/security/sdr;1"].
>+              getService(Ci.nsISecretDecoderRing);
>+    sdr.logoutAndTeardown();
>+
>+    // clear plain HTTP auth sessions
>+    let authMgr = Cc['@mozilla.org/network/http-auth-manager;1'].
>+                  getService(Ci.nsIHttpAuthManager);
>+    authMgr.clearAll();
>+
>+    let ss = Cc["@mozilla.org/browser/sessionstore;1"].
>+             getService(Ci.nsISessionStore);
>+    if (this.privateBrowsingEnabled) {
>+      // first, give extensions a chance to provide their own UI here
>+      let decision = Cc["@mozilla.org/supports-PRUint32;1"].
>+                     createInstance(Ci.nsISupportsPRUint32);
>+      decision.data = Ci.nsIBrowserGlue.PRIVATEBROWSING_NODECISION;
>+      this._obs.notifyObservers(decision, "private-browsing-enter", null);
>+      switch (decision.data) {
>+      case Ci.nsIBrowserGlue.PRIVATEBROWSING_KEEPSESSION:
>+      case Ci.nsIBrowserGlue.PRIVATEBROWSING_CLOSESESSION:
>+        this._sessionDecision = decision.data;
>+        break;
>+      default:
>+        // if no extension handles the decision, let the browser glue component decide
>+        let browserGlue = Cc["@mozilla.org/browser/browserglue;1"].
>+                          getService(Ci.nsIBrowserGlue);
>+        this._sessionDecision = (browserGlue.decidePrivateBrowsingSession() ==
>+                                 Ci.nsIBrowserGlue.PRIVATEBROWSING_KEEPSESSION) ?
>+                                Ci.nsIBrowserGlue.PRIVATEBROWSING_KEEPSESSION :
>+                                Ci.nsIBrowserGlue.PRIVATEBROWSING_CLOSESESSION;
>+        break;
>+      }

as noted, simplify the hell out of this, we don't want this UI, but we need the pref to work around unit test weakness.

>+      // save the whole browser state in order to restore all windows/tabs later
>+      if (this._sessionDecision == Ci.nsIBrowserGlue.PRIVATEBROWSING_CLOSESESSION &&
>+          this._savedBrowserState == null) {
>+        this._savedBrowserState = ss.getBrowserState();

we should just restore from disk, instead of keeping all of this stashed in memory.  you're being paranoid. :)


>+  _closeAllWindows: function PBS__closeAllWindows() {
>+    let windowMediator = Cc["@mozilla.org/appshell/window-mediator;1"].
>+                         getService(Ci.nsIWindowMediator);
>+    let windowsEnum = windowMediator.getEnumerator("navigator:browser");
>+
>+    // We want to close all windows in reverse order, but since
>+    // nsISimpleEnumerator can't do backwards iterations, a little
>+    // recursive helper function is used.
>+    this._closeAllWindowsInternal(windowsEnum);
>+  },

I think we want back to front.

you want windowMediator.getZOrderDOMWindowEnumerator("navigator:browser", false) on Windows, other platforms are busted. :(

>+  _closeAllWindowsInternal: function PBS__closeAllWindowsInternal(windowsEnum) {
>+    if (windowsEnum.hasMoreElements()) {
>+      let window = windowsEnum.getNext();
>+      this._closeAllWindowsInternal(windowsEnum);
>+      window.close();
>+    }
>+  },

use win instead of window, window has special meaning in JS, and we should avoid using it as a generic var.

>+    try {

>+    } finally {
>+      this._alreadyChangingMode = false;
>+    }
>+  }
>+};

probably worth a reportError call in a catch block if there's an exception thrown in the try block.

I didn't look at the tests yet, but please roll those into the unit test changes needed because we're changing the notifications they'll need

I'll look at this again when you've done these changes.
Attachment #344819 - Flags: review?(mconnor) → review-
Comment on attachment 343975 [details] [diff] [review]
[for review] DOM storage v1

I'm fine with the storage patch, assuming the behavior (security exception on access) is the desired behavior.
Attachment #343975 - Flags: review?(dcamp) → review+
Blocks: 462218
(In reply to comment #468)
> as discussed on IRC, this is because Cc isn't defined when you call this.  We
> really need to clean up Cc, its this weird sorta-const that isn't the same as
> Ci and Cu.

Moving the init code to delayedStartup did fix this problem...

> >+    } else if (aTopic == "quit-application") {
> 
> don't you have enough nits on this yet?

Nah, can't get enough of them! ;-)

> >+gPrivateBrowsingUI.init();
> 
> why is this called this early?  there's no need to call it there...

I moved it to delayedStartup, and merged init and initUI as well.

> >+  decidePrivateBrowsingSession: function()
> >+  {
> 
> >+    return result;
> >+  },
> 
> we don't want this anymore.  Its a whole bunch of code that enables a hidden
> pref.  have the pref for unit tests, kill the UI/prompt.

Killed.

> as noted, simplify the hell out of this, we don't want this UI, but we need the
> pref to work around unit test weakness.

Done.

> >+      // save the whole browser state in order to restore all windows/tabs later
> >+      if (this._sessionDecision == Ci.nsIBrowserGlue.PRIVATEBROWSING_CLOSESESSION &&
> >+          this._savedBrowserState == null) {
> >+        this._savedBrowserState = ss.getBrowserState();
> 
> we should just restore from disk, instead of keeping all of this stashed in
> memory.  you're being paranoid. :)

As discussed on IRC, I'm moving this into a followup: bug 462218.

> I think we want back to front.
> 
> you want windowMediator.getZOrderDOMWindowEnumerator("navigator:browser",
> false) on Windows, other platforms are busted. :(

Done, but I'm not really sure if I've managed to do what's desired here...

> use win instead of window, window has special meaning in JS, and we should
> avoid using it as a generic var.

Oops, that was way embarrassing to slip my attention!

> probably worth a reportError call in a catch block if there's an exception
> thrown in the try block.

Done.

> I didn't look at the tests yet, but please roll those into the unit test
> changes needed because we're changing the notifications they'll need

I will do this in another patch which I'll attach shortly.

> I'll look at this again when you've done these changes.

I also removed all of the UI pieces for this patch to reduce the amount of rework done on the UI code.
Attachment #344819 - Attachment is obsolete: true
Attachment #345361 - Flags: review?(mconnor)
This patch handles the API change for all of the unit tests: drop the usage of private-browsing-enter, and use the browser.privatebrowsing.keep_current_session prefs instead.

Nothing major here, the only thing that I noted was that the notification change in attachment 343974 [details] [diff] [review] wasn't reflected in the unit test, so I went ahead and corrected it as well.
Attachment #345377 - Flags: superreview?(mconnor)
Attachment #345377 - Flags: review?(mconnor)
Comment on attachment 345377 [details] [diff] [review]
[checked in] test api changes v1 (Checkin: comment 503)

looks good, r=mconnor
Attachment #345377 - Flags: superreview?(mconnor)
Attachment #345377 - Flags: superreview+
Attachment #345377 - Flags: review?(mconnor)
Attachment #345377 - Flags: review+
Attachment #345361 - Flags: review?(mconnor) → review+
Comment on attachment 345361 [details] [diff] [review]
[checked in] service v2 (Checkin: comment 502)

So, again, to complain:

} else 

should be

}
else

>diff --git a/browser/base/content/browser.js b/browser/base/content/browser.js

these bits are all part of the UI bits, so don't need to be in this patch at all.


>+#ifndef XP_WIN
>+#define BROKEN_WM_Z_ORDER
>+#endif
>+
>+  _closeAllWindows: function PBS__closeAllWindows() {
>+    let windowMediator = Cc["@mozilla.org/appshell/window-mediator;1"].
>+                         getService(Ci.nsIWindowMediator);
>+#ifdef BROKEN_WM_Z_ORDER
>+    let windowsEnum = windowMediator.getEnumerator("navigator:browser");
>+#else
>+    let windowsEnum = windowMediator.getZOrderDOMWindowEnumerator("navigator:browser", false);
>+#endif

why not just #ifdef XP_WIN with a comment here?  The extra define isn't helpful, if its Windows only...

>+  /**
>+   * Enter or leave private browsing mode.
>+   */
>+  set privateBrowsingEnabled PBS_set_privateBrowsingEnabled(val) {
>+    if (this._alreadyChangingMode)
>+      throw Cr.NS_ERROR_FAILURE;

comments as to how this happens/why its necessary would be good

we're getting there! r=mconnor
(In reply to comment #473)

> >+#ifndef XP_WIN
> >+#define BROKEN_WM_Z_ORDER
> >+#endif
> >+
> >+  _closeAllWindows: function PBS__closeAllWindows() {
> >+    let windowMediator = Cc["@mozilla.org/appshell/window-mediator;1"].
> >+                         getService(Ci.nsIWindowMediator);
> >+#ifdef BROKEN_WM_Z_ORDER
> >+    let windowsEnum = windowMediator.getEnumerator("navigator:browser");
> >+#else
> >+    let windowsEnum = windowMediator.getZOrderDOMWindowEnumerator("navigator:browser", false);
> >+#endif
> 
> why not just #ifdef XP_WIN with a comment here?  The extra define isn't
> helpful, if its Windows only...

Actually it is because it is the standard define name for this kind of hack around other places in the codebase. If you leave it as XP_WIN then:

a) its not clear why the hack is needed
b) it makes it harder to remove this hack by looking for it in MXR if other platforms get this implemented in the future.
(In reply to comment #474)
> Actually it is because it is the standard define name for this kind of hack
> around other places in the codebase. If you leave it as XP_WIN then:
> 
> a) its not clear why the hack is needed
> b) it makes it harder to remove this hack by looking for it in MXR if other
> platforms get this implemented in the future.

I tend to agree with Michael Ventnor here.  mconnor: do we really want to get rid of this define?

(In reply to comment #473)
> >diff --git a/browser/base/content/browser.js b/browser/base/content/browser.js
> 
> these bits are all part of the UI bits, so don't need to be in this patch at
> all.

Have you reviewed these changes or should I add them to the UI patch for review?
Depends on: 462573
Blocks: 462579
bah, just leave the define for now.  We need to fix that API.

I've reviewed it all, you are clear for takeoff.
(In reply to comment #476)
> bah, just leave the define for now.  We need to fix that API.

OK.

> I've reviewed it all, you are clear for takeoff.

Actually I think it would be better to wait up a bit for the remaining parts so that we can land it all together.

The remaining parts:

1. A decision needs to be made how we want to handle the errors for the DOM storage module.  Here is my suggestion: we take this as it is right now (throwing a security error when inside the private mode) for Beta 2, and I'll file a followup to work on changing it to use a memory backed DB instead of the usual disk DB, just like what we do for cookies now.  I'll provide a patch for that bug in time to make it into the final 3.1 release.

2. The download manager tests have been delayed for so long.  I'll try to work on it and get something ready by tomorrow.  Shawn: if you can help with the test plan, it'd be great.  Otherwise I'll give it a shot myself.

3. I've just observed that the places unit test has started to fail.  I need to investigate this further, but I suspect it has something to do with moving stuff to background threads in the fsync work recently landed.  I think the culprit is that the bookmarks are inserted lazily, which happens too late for the places unit test to catch.  Any guidance in this regard is highly appreciated.


We're so close now!  :-)
Attached patch [for review] sessionstore v5 (obsolete) — Splinter Review
Simon: this patch only changes the (a) unit test to use nsITimer instead of setTimeout based on mconnor's suggestion.  I was not sure whether this change requires another review, but I thought I'd ask in order to be safe.
Attachment #344732 - Attachment is obsolete: true
Attachment #345808 - Flags: review?(zeniko)
Comment on attachment 345808 [details] [diff] [review]
[for review] sessionstore v5

Isn't the "private-browsing-enter" notification supposed to be replaced by a simple pref? Furthermore: Have you talked to sdwilsh about why setTimeout isn't good enough for what we need (all my other tests use it - and they work just fine)?
Depends on: 462639
(In reply to comment #479)
> (From update of attachment 345808 [details] [diff] [review])
> Isn't the "private-browsing-enter" notification supposed to be replaced by a
> simple pref?

Yes, it is.  I fixed all of the unit tests in a single go as attachment 345377 [details] [diff] [review] which got reviewed.  This patch is applied on top of the session store patch in my queue <http://hg.mozilla.org/users/ehsan.akhgari_gmail.com/mq/> so the sessionstore part will also use that pref when I land this.

> Furthermore: Have you talked to sdwilsh about why setTimeout isn't
> good enough for what we need (all my other tests use it - and they work just
> fine)?

Not yet.  Just tried to find him on IRC to no avail.  But is there anything which would prevent us from using nsITimer over setTimeout?  Either way it's not a big deal to me, unless I hear from sdwilsh that something's awfully wrong with setTimeout...
Attachment #345808 - Flags: review?(zeniko) → review+
Comment on attachment 345808 [details] [diff] [review]
[for review] sessionstore v5

If Shawn doesn't object, I'd prefer the setTimeout patch - otherwise this one is fine by me as well, if you just directly pass the function to initWithCallback (and prefix the function attributes' names with an a):

> timer.initWithCallback(function(aTimer) {
>   // ...
> }, 3000, Ci.nsITimer.TYPE_ONE_SHOT);
Blocks: 462106
No longer depends on: 462106
No longer blocks: 460346
Depends on: 460346
Drive-by: Isn't there away to do this *without* a timer?

Timers are generally frowned up because they're failure prone when a test box is running slowly or someone's debugging under valgrind. And our tests already take too long to run, so adding unneeded delays (6 seconds, in this test) makes things worse.
That would require getting the sessionstore component to record its data on command, instead of at an interval.  Simon: if that's possible, then I'm all for it.
Comment on attachment 345808 [details] [diff] [review]
[for review] sessionstore v5

So, I've had a look at a simplified version of your patch and don't get the test to fail even without invoking private browsing... You'll need to get a new file reference for an updated lastModifiedTime so that it could be different at all.

Then, the first of the two timers isn't really needed at all, and the second one can be replaced by

>+ gPrefService.setIntPref("browser.sessionstore.interval", 0);

which triggers an immediate save operation. Then please make sure that without enabling PB mode the test indeed fails and only passes with PB mode enabled first.

In fact, I'd prefer such a test to happen right before entering PB mode so that you can be sure that the test actually means something. Just don't forget to reset the .interval pref before entering PB mode so that repeatedly setting it to 0 also repeatedly sends a pref-change notification.
Attachment #345808 - Flags: review+ → review-
Attached patch [for review] downloadmgr v1 (obsolete) — Splinter Review
Download manager patch with a unit test, ready for review.

Asking sdwilsh to review the download manager service parts, and mconnor to review the rest of the toolkit changes.
Attachment #345902 - Flags: review?(sdwilsh)
Attachment #345902 - Flags: review?(mconnor)
Attached patch [for review] sessionstore v6 (obsolete) — Splinter Review
(In reply to comment #484)

Simon: I tried to address all of your comments.  Also I ran both tests without entering/leaving the private mode to make sure they fail in that case (and indeed they do).  Please let me know if there are more issues to work out.
Attachment #345808 - Attachment is obsolete: true
Attachment #345907 - Flags: review?(zeniko)
The latest fsync work in Places broke the unit test for Places which is already checked in.  Here's a fix for the unit test to make it wait for sync notifications, and also fix an issue where the places query object returned both history and bookmark items (I'm not sure why), but anyway the current behavior is correct (we make two queries, one for history, the other for bookmarks).
Attachment #345927 - Flags: review?(dietrich)
Comment on attachment 345907 [details] [diff] [review]
[for review] sessionstore v6

>+  let ss_interval = gPrefService.getIntPref("browser.sessionstore.interval");
>+  gPrefService.setIntPref("browser.sessionstore.interval", 0);

Please verify that sessionstore.js's lastModifiedTime differs before and after this point.

>+    // enter private browsing mode
>+    pb.privateBrowsingEnabled = true;

It should differ once again after this point.

>+    gPrefService.setIntPref("browser.sessionstore.interval", 0);

This is a no-op if the pref is already set to 0 (which it is). Just drop the line.

>+        // record the timestamp of private browsing session - sessionstore.js
>+        gPrefService.setIntPref("browser.sessionstore.interval", 0);

Still a no-op. Reset the pref before this line for it to have any effect.

>+            flag = (x.title==y.title && x.url==y.url); // comparing titles

While I'm at it... nit: No spaces around operators.
Attached patch [for review] sessionstore v7 (obsolete) — Splinter Review
(In reply to comment #488)
> (From update of attachment 345907 [details] [diff] [review])

All comments addressed.
Attachment #345907 - Attachment is obsolete: true
Attachment #345953 - Flags: review?(mconnor)
Attachment #345907 - Flags: review?(zeniko)
Comment on attachment 345953 [details] [diff] [review]
[for review] sessionstore v7

Oh, I guess I'm too used to ask mconnor for reviews!  :-)  Meant Simon, of course...
Attachment #345953 - Flags: review?(mconnor) → review?(zeniko)
Comment on attachment 345953 [details] [diff] [review]
[for review] sessionstore v7

>+  _backupRecentlyClosedTabs: function sss_backupRecentlyClosedTabs() {

BTW: Can we just drop these now that users won't have the option of keeping their current windows open?

>+  function test(aLambda) {

As you'll need another revision round, anyway, could you please just drop this function. You no longer use it as intended... ;-)

>+    if (file.exists())

Out of curiosity: Can the file be missing at all?

>+    gPrefService.setIntPref("browser.sessionstore.interval", ss_interval);
>+    gPrefService.setIntPref("browser.sessionstore.interval", 0);
>+    let startPBModeTimeStamp = getSessionstorejsModificationTime();

You'll get the very same timestamp without changing the pref.

>+        function serializeClosedTabData(array) {
>+          array.sort(function (a, b) {
>+            return a.pos - b.pos;
>+          });

What's the need for sorting? Wouldn't this rather lead to two different closed tab lists being considered equal?

>+                return aPrevious + aCurrent.url + "#" + aCurrent.title + "#";

What was wrong with the comparison you had in revision 6 of this patch? If you want to stick to this one, please use a separator that isn't as likely to be contained in either title or URL (such as "\n" or even better "\0").
Comment on attachment 345927 [details] [diff] [review]
[checked in] places test fix v1 (Checkin: comment 504)

looks ok, though for readability i'd prefer that you separate out and name the observers, instead of having them nested and anonymous. r=me w/ that change.
Attachment #345927 - Flags: review?(dietrich) → review+
(In reply to comment #491)
> (From update of attachment 345953 [details] [diff] [review])
> >+  _backupRecentlyClosedTabs: function sss_backupRecentlyClosedTabs() {
> 
> BTW: Can we just drop these now that users won't have the option of keeping
> their current windows open?

OK, I won't resist it this time!  ;-)

Done.

> >+  function test(aLambda) {
> 
> As you'll need another revision round, anyway, could you please just drop this
> function. You no longer use it as intended... ;-)

Done.

> >+    if (file.exists())
> 
> Out of curiosity: Can the file be missing at all?

Yes, if you choose to run only the tests in browser/components/sessionstore, this test is run as the first test, before the SS component has kicked in to write sessionstore.js for the first time.

BTW, there is an annoying bug in one of the sessionstore tests which prevents one from running only tests in that directory.  It causes the whole window to be closed along the way.  Are you aware of this or should I file a bug?

> You'll get the very same timestamp without changing the pref.

Done.

> What's the need for sorting? Wouldn't this rather lead to two different closed
> tab lists being considered equal?

Hmmm, yes I think you're right, but it won't matter anyway now, since we killed the code which was supposed to make this work!

> What was wrong with the comparison you had in revision 6 of this patch? If you
> want to stick to this one, please use a separator that isn't as likely to be
> contained in either title or URL (such as "\n" or even better "\0").

The previous one had flaws, such as failing when both arrays were empty, but like I said it doesn't matter now, because all the tests regarding recently closed tabs have been removed now.
Attachment #345953 - Attachment is obsolete: true
Attachment #345990 - Flags: review?(zeniko)
Attachment #345953 - Flags: review?(zeniko)
Comment on attachment 345990 [details] [diff] [review]
[checked in] sessionstore v8 (Checkin: comment 505)

(In reply to comment #493)
> this test is run as the first test, before the SS component has
> kicked in to write sessionstore.js for the first time.

So, you're still using a build without the fix for bug 395488? ;-)

> BTW, there is an annoying bug in one of the sessionstore tests which
> prevents one from running only tests in that directory.

Doesn't happen on my machine. Please file a bug, indicating the offender.

Thanks again for your patience on this. r+=me
Attachment #345990 - Flags: review?(zeniko) → review+
Will the next bits be landing shortly?
(In reply to comment #494)
> > BTW, there is an annoying bug in one of the sessionstore tests which
> > prevents one from running only tests in that directory.
> 
> Doesn't happen on my machine. Please file a bug, indicating the offender.

Done: bug 462794.

> Thanks again for your patience on this. r+=me

Thank you for taking the time to review this over and over again!

(In reply to comment #495)
> Will the next bits be landing shortly?

Yeah, as soon as I get a review on the download manager changes, the whole thing (minus the DOM storage part) will land.
Comment on attachment 345902 [details] [diff] [review]
[for review] downloadmgr v1

>diff --git a/toolkit/components/downloads/src/nsDownloadManager.cpp b/toolkit/components/downloads/src/nsDownloadManager.cpp

>-  // The following three AddObserver calls must be the last lines in this function,
>+  // The following nine AddObserver calls must be the last lines in this function,

this made me laugh, then cringe a little.

looks good to me, but really want to get sdwilsh's stamp on this ASAP.
Attachment #345902 - Flags: review?(mconnor) → review+
Blocks: mozpb
Blocks: 251677
Blocks: 460895
Attachment #345902 - Flags: review?(sdwilsh) → review-
Comment on attachment 345902 [details] [diff] [review]
[for review] downloadmgr v1

>diff --git a/toolkit/components/downloads/src/nsDownloadManager.cpp b/toolkit/components/downloads/src/nsDownloadManager.cpp
>+  nsCOMPtr<nsIPrivateBrowsingService> pbs =
>+    do_GetService(NS_PRIVATE_BROWSING_SERVICE_CONTRACTID);
>+  if (pbs) {
>+    pbs->GetPrivateBrowsingEnabled(&mInPrivateBrowsing);
This function returns something, but you do not check that value.  If you do not care about the return value, please cast to void (you do it elsewhere, but I'm not going to comment on it anymore).


>+    if (mInPrivateBrowsing)
>+      Observe(nsnull, NS_PRIVATE_BROWSING_SWITCH_TOPIC,
>+        NS_LITERAL_STRING(NS_PRIVATE_BROWSING_ENTER).get());
I don't like using the Observe method like this (we aren't observing anything).  Please break out the code that does the work for that and just call that method.

>-  // The following three AddObserver calls must be the last lines in this function,
>+  // The following nine AddObserver calls must be the last lines in this function,
nit: just remove three.  people clearly do not update this number, and it's not useful anyway.

>@@ -2027,16 +2039,54 @@ nsDownloadManager::Observe(nsISupports *
>       (void)pref->GetIntPref(PREF_BDM_RESUMEONWAKEDELAY, &resumeOnWakeDelay);
> 
>     // Wait a little bit before trying to resume to avoid resuming when network
>     // connections haven't restarted yet
>     mResumeOnWakeTimer = do_CreateInstance("@mozilla.org/timer;1");
>     if (resumeOnWakeDelay >= 0 && mResumeOnWakeTimer) {
>       (void)mResumeOnWakeTimer->InitWithFuncCallback(ResumeOnWakeCallback,
>         this, resumeOnWakeDelay, nsITimer::TYPE_ONE_SHOT);
>+    }
>+  } else if (strcmp(aTopic, NS_PRIVATE_BROWSING_REQUEST_TOPIC) == 0 && currDownloadCount) {
nit: line wrapping at 80 columns please
nit: I know this method is not consistent, but please have the else if on a new line after the brace.

>+    } else if (NS_LITERAL_STRING(NS_PRIVATE_BROWSING_LEAVE).Equals(aData)) {
ditto

>+  } else if (strcmp(aTopic, NS_PRIVATE_BROWSING_SWITCH_TOPIC) == 0) {
ditto

>+    } else if (NS_LITERAL_STRING(NS_PRIVATE_BROWSING_LEAVE).Equals(aData)) {
ditto

>diff --git a/toolkit/components/downloads/test/unit/test_privatebrowsing.js b/toolkit/components/downloads/test/unit/test_privatebrowsing.js
>+function get_PBSvc() {
>+  try {
>+    _PBSvc = Components.classes["@mozilla.org/privatebrowsing;1"].
>+             getService(Components.interfaces.nsIPrivateBrowsingService);
>+    return _PBSvc;
>+  } catch (e) {}
>+  return null;
>+}
>+
>+let _DMSvc = null;
>+function get_DownloadManager() {
>+  if (_DMSvc)
>+    return _DMSvc;
>+
>+  return _DMSvc = Components.classes["@mozilla.org/download-manager;1"].
>+                  getService(Components.interfaces.nsIDownloadManager);
>+}
This doesn't follow convention of the rest of the download manager test files.  We use Cc, Ci, etc.  Also, if you want to use a lazy getter, please just use a smart getter:
this.__defineGetter__("dm", function() {
  delete this.dm;
  return this.dm = Cc["@mozilla.org/download-manager;1"].
                   getService(Ci.nsIDownloadManager);
});

>+function is_download_available1(id, src, dst, name) {
javadoc style comments, and please prefix pArameters with a.

>+function is_download_available2(id, src, dst, name) {
ditto

>+function run_test() {
>+  let pb = get_PBSvc();
>+  if (pb) { // Private Browsing might not be available
It makes the code a lot cleaner if you just return here instead of having everything inside an if block.

General comments for the rest of the test:
You use a few things like variable_nameNumber just because you have more than one.  You can make the code easier to follow by using a let block:
let (timer = ...) {
  // use the variable
}

In at least one place you wrap a bunch of code in a try block, and in the catch, you just report a test failure.  This is unnecessary since the test will fail when an exception is thrown anyway.  Please remove it.

It looks like you copied the code for addDownload in head_download_manager.js with some minor modifications.  Any reason why you didn't modify that function and make it more flexible instead of copying and pasting code?

You have a long switch statement inside a long switch statement.  Please pull out the second one into a method so it's easier to see what is going on.

It would also be really nice to have a comment at the top of the file explaining what, exactly, this test is doing because the code is hard to follow (it's very very long).

I know it seems like I'm being picky on the test, but if it ever starts to fail, I want people to be able to understand why instead so they have a chance to fix things.

>diff --git a/toolkit/mozapps/downloads/content/downloads.js b/toolkit/mozapps/downloads/content/downloads.js
> ////////////////////////////////////////////////////////////////////////////////
> //// Utility Functions
There are, sadly, two places where we have utility functions in this file.  Please add them to the bottom of the file, not the one at the top.

>+function initStmt()
initStatement please, and include a javadoc comment (and include why we have it in a method)

>+{
>+  if (gStmt) {
>+    gStmt.reset();
>+    gStmt.finalize();
just call finalize.

>@@ -504,16 +517,24 @@ let gDownloadObserver = {
>+      case "private-browsing":
>+        if (aData == "enter" || aData == "exit") {
>+          setTimeout(function() {
>+            initStmt();
>+            buildDownloadList(true);
>+          }, 0);
>+        }
Why are we doing this in a setTimeout?  If there is a good reason, we should have a comment explaining why.
(In reply to comment #498)
> >+    if (mInPrivateBrowsing)
> >+      Observe(nsnull, NS_PRIVATE_BROWSING_SWITCH_TOPIC,
> >+        NS_LITERAL_STRING(NS_PRIVATE_BROWSING_ENTER).get());
> I don't like using the Observe method like this (we aren't observing anything).
>  Please break out the code that does the work for that and just call that
> method.

Done.  I added both OnEnterPrivateBrowsingMode and OnLeavePrivateBrowsingMode for parity.

> General comments for the rest of the test:
> You use a few things like variable_nameNumber just because you have more than
> one.  You can make the code easier to follow by using a let block:
> let (timer = ...) {
>   // use the variable
> }

As discussed on IRC, this wasn't necessary because I only had one timer!  ;-)

> In at least one place you wrap a bunch of code in a try block, and in the
> catch, you just report a test failure.  This is unnecessary since the test will
> fail when an exception is thrown anyway.  Please remove it.

Done.  For future reference, those were the places which we were running from a timer, where an exception is not caught by the unit test framework in the usual way.  Without them, the test will time out if something throws there.

> It looks like you copied the code for addDownload in head_download_manager.js
> with some minor modifications.  Any reason why you didn't modify that function
> and make it more flexible instead of copying and pasting code?

Done using the object-as-parameter approach Shawn suggested on IRC.  One caller which did use the old-style parameter was patched as well.

> I know it seems like I'm being picky on the test, but if it ever starts to
> fail, I want people to be able to understand why instead so they have a chance
> to fix things.

No, not at all!  I really appreciate all time time and energy you put into this.  The new iteration is way better than the previous one in terms of readability and maintainability.

> >@@ -504,16 +517,24 @@ let gDownloadObserver = {
> >+      case "private-browsing":
> >+        if (aData == "enter" || aData == "exit") {
> >+          setTimeout(function() {
> >+            initStmt();
> >+            buildDownloadList(true);
> >+          }, 0);
> >+        }
> Why are we doing this in a setTimeout?  If there is a good reason, we should
> have a comment explaining why.

Hmmm, I couldn't remember why I did that, but I think it should've been a mistake.  I removed the setTimeout call and tested the patch again to make sure everything's working properly.

All other comments and nits fixed as well.
Attachment #345902 - Attachment is obsolete: true
Attachment #346103 - Flags: review?(sdwilsh)
Attachment #346103 - Flags: review?(sdwilsh) → review+
Comment on attachment 346103 [details] [diff] [review]
[checked in] downloadmgr v2 (Checkin: comment 506)

r=sdwilsh
Whiteboard: PRD:SPI-002a → PRD:SPI-002a [pb-ready-for-landing]
Private browsing code and dependencies landed (thanks Shawn).  I'll update the bugs with changeset links and close it if everything stays green in about three hours.
Depends on: 462986
Comment on attachment 345361 [details] [diff] [review]
[checked in] service v2 (Checkin: comment 502)

http://hg.mozilla.org/mozilla-central/rev/12487541851b
Attachment #345361 - Attachment description: [for review] service v2 → [checked in] service v2 (Checkin: comment 502)
Comment on attachment 345377 [details] [diff] [review]
[checked in] test api changes v1 (Checkin: comment 503)

http://hg.mozilla.org/mozilla-central/rev/a6712f1f6c4e
Attachment #345377 - Attachment description: [for review] test api changes v1 → [checked in] test api changes v1 (Checkin: comment 503)
Comment on attachment 345927 [details] [diff] [review]
[checked in] places test fix v1 (Checkin: comment 504)

http://hg.mozilla.org/mozilla-central/rev/c588d41a49e5
Attachment #345927 - Attachment description: [for review] places test fix v1 → [checked in] places test fix v1 (Checkin: comment 504)
Comment on attachment 345990 [details] [diff] [review]
[checked in] sessionstore v8 (Checkin: comment 505)

http://hg.mozilla.org/mozilla-central/rev/743136ef83d9
Attachment #345990 - Attachment description: [for review] sessionstore v8 → [checked in] sessionstore v8 (Checkin: comment 505)
Comment on attachment 346103 [details] [diff] [review]
[checked in] downloadmgr v2 (Checkin: comment 506)

http://hg.mozilla.org/mozilla-central/rev/f67cc64a3517
Attachment #346103 - Attachment description: [for review] downloadmgr v2 → [checked in] downloadmgr v2 (Checkin: comment 506)
OK, I think this is now worth marking RESOLVED FIXED finally!  :-)

The only test failure was filed as 462986 which already has a patch to fix the issue.  The remaining work will be handled in separate bugs.  I'll also file a bug to discuss what we want to do with the DOM Storage module for the final 3.1 release.
Status: ASSIGNED → RESOLVED
Closed: 16 years ago
Flags: in-testsuite? → in-testsuite+
Keywords: dev-doc-needed
Resolution: --- → FIXED
Whiteboard: PRD:SPI-002a [pb-ready-for-landing] → PRD:SPI-002a
Target Milestone: --- → Firefox 3.1b2
Depends on: 463000
Depends on: 463021
Depends on: 463022
Comment on attachment 346103 [details] [diff] [review]
[checked in] downloadmgr v2 (Checkin: comment 506)

+enterPrivateBrowsingCancelDownloadsAlertMsg=If you enter the Private Browsing mode now, 1 download will be canceled. Are you sure you want to enter the Private Browsing mode?
+enterPrivateBrowsingCancelDownloadsAlertMsgMultiple=If you enter the Private Browsing mode now, %S downloads will be canceled. Are you sure you want to enter the Private Browsing mode?

Could anyone tell why the built-in support for plural forms wasn't used here?
(In reply to comment #508)
> Could anyone tell why the built-in support for plural forms wasn't used here?

Because Ehsan hates localizers!

Really though, better in general to just file a new bug when you notice someone like this, and CC the relevant people. :)
Depends on: 463102
(In reply to comment #508)
> Could anyone tell why the built-in support for plural forms wasn't used here?

That's an oversight on my part, filed bug 463102 on that.

(In reply to comment #509)
> Because Ehsan hates localizers!

That'd be weird, because I'm also a localizer myself.  ;-)
Depends on: 463183
Blocks: 463188
Blocks: 463189
Blocks: 463202
Why are all windows closed when I go into private browsing mode? I would expect the window where I toggle the private browsing mode on, to go into private browsing mode.
(In reply to comment #511)
> Why are all windows closed when I go into private browsing mode? I would expect
> the window where I toggle the private browsing mode on, to go into private
> browsing mode.

Private browsing mode is global, not per window/tab.  Because we didn't have any way to associate only the newly opened window with the private browsing mode, we decided to close the current session to make the confusion about which tabs/pages were opened in private mode and which ones were not minimal.
Depends on: 463399
Blocks: 463400
Blocks: 463132
Depends on: 463256
Ehsan: are we persisting the "Save As..." location?  I completely forgot about that being something that we need to clear.
(In reply to comment #513)
> Ehsan: are we persisting the "Save As..." location?  I completely forgot about
> that being something that we need to clear.

We don't handle it currently.  Can you please file a new bug and assign it to me to make sure I won't forget to work on it?  Thanks!
Depends on: 463578
Depends on: 463607
Flags: blocking-firefox3.1? → blocking-firefox3.1+
Blocks: 463692
Depends on: 463885
(In reply to comment #513)
> Ehsan: are we persisting the "Save As..." location?  I completely forgot about
> that being something that we need to clear.

Filed bug 463888 about this.
Depends on: 463888
https://litmus.mozilla.org/show_test.cgi?id=7394 added to Litmus.
Flags: in-litmus? → in-litmus+
Depends on: 464792
Attachment #343975 - Flags: superreview?(jonas)
Depends on: 464962
Depends on: 468301
No longer depends on: 463399
Depends on: 468879
Hello,

Today I was using Firefox/3.1b2 and took notice that while in PB mode, shared objects/temp are still being serialized to disk (this is an Adobe Flash player feature) - as a result: page visits with flash that create shared objects are displayed clearly %appdata% profile.

i.e., C:\Users\xxx\AppData\Roaming\Macromedia\Flash Player\#SharedObjects\4ZW9JGZ5\tdcanadatrust.com

Where tdcanadatrust.com was the site I visited under PB mode. Not so private now :)

I tested with Flash 10, FF 3.1b2 on Vista.

Is this something we should address?
I am not sure if this is even a flash issue even.
I believe it is a feature called "Flash Cookies".
(In reply to comment #517)
> Hello,
> Today I was using Firefox/3.1b2 and took notice that while in PB mode, shared
> objects/temp are still being serialized to disk (this is an Adobe Flash player
> feature) - as a result: page visits with flash that create shared objects are
> displayed clearly %appdata% profile.
> i.e., C:\Users\xxx\AppData\Roaming\Macromedia\Flash
> Player\#SharedObjects\4ZW9JGZ5\tdcanadatrust.com
> Where tdcanadatrust.com was the site I visited under PB mode. Not so private
> now :)
> I tested with Flash 10, FF 3.1b2 on Vista.
> Is this something we should address?

File a new bug if it isn't removed when you exit private browsing mode.  Also make it dependent on this bug or at the very least post the bug number here.

You might also be needing these bugs to be fixed: bug 468879 and bug 468877
Bug 46887 pretty much is the same idea at hand.
Aaron: we are not going to provide plugin-specific protection for Flash, etc inside the Mozilla code.  Bug 468877 is going to help with this issue if plugin vendors opt in to implement it.  Bug 468879 will enable disabling all plugins in the private browsing mode, but that will only work for privacy conscious users who don't care if their private browsing session can't use plugins.

And of course, extensions such as FlashBlock and NoScript can respond to private browsing mode changes to implement some amount of protection against this.
No longer depends on: 469212
Mass moving of all Firefox::General private browsing bugs to Firefox::Private Browsing.
Component: General → Private Browsing
QA Contact: general → private.browsing
Everything is completed on this bug. No perf regressions with the checked-in patch. Now some weeks later we can mark this bug safely as verified. All remaining issues will be covered on their own bugs.
Status: RESOLVED → VERIFIED
No longer blocks: 462106
Depends on: 462106
fwiw, comment 220's was addressed by bug 447648
fwiw, comment 220 was addressed by bug 447648
Depends on: 470241
I've found a new bug. When I want to turn on Private/Porno Mode I have to use two hands (crtl+shift+p). Sometimes this is impossible, you know...
(In reply to comment #530)
> I've found a new bug. When I want to turn on Private/Porno Mode I have to use
> two hands (crtl+shift+p). Sometimes this is impossible, you know...

Then use:  Tools->Start Private browsing
(In reply to comment #531)
> (In reply to comment #530)
> > I've found a new bug. When I want to turn on Private/Porno Mode I have to use
> > two hands (crtl+shift+p). Sometimes this is impossible, you know...
> Then use:  Tools->Start Private browsing
You can also use keyconfig (http://forums.mozillazine.org/viewtopic.php?f=48&t=72994&hilit=key) to change the access keys or toggle private browsing (https://addons.mozilla.org/en-US/firefox/addon/9517) to have a toolbar button to toggle between private and regular browsing modes.
(In reply to comment #530)
> I've found a new bug. When I want to turn on Private/Porno Mode I have to use
> two hands (crtl+shift+p). Sometimes this is impossible, you know...

Actually, you can type ctrl-shift-p with the right hand : almost all keyboards have ctrl and shift keys on the right hand side too. 

And may I congratulate you with the funniest bug report of the year :-)
(In reply to comment #533)
> Actually, you can type ctrl-shift-p with the right hand : almost all keyboards
> have ctrl and shift keys on the right hand side too.

Obviously he's right-handed... :P  But yes, he could still use his left hand to type the ctrl-shift-p assuming the keyboard supports it.
Example: I use Windows with boot Camp on a mac computer
I don't have a ctrl on the right side

and: ctrl+shift+p is normally 'page setup', so may be not a good combination
there is a pretty interesting graphic about the kind of interest and impact this feature has had over in https://bug503576.bugzilla.mozilla.org/attachment.cgi?id=388008
No longer blocks: 489585
Depends on: 489585
No longer blocks: 456884
Blocks: 456884
No longer depends on: 582474
Depends on: 717871
No longer blocks: 460895
Depends on: 892487
You need to log in before you can comment on or make changes to this bug.