Closed Bug 62173 Opened 24 years ago Closed 3 years ago

StringBundle cache-flushed notification

Categories

(Core :: Internationalization, defect, P3)

defect

Tracking

()

RESOLVED WONTFIX
Future

People

(Reporter: tao, Assigned: smontagu)

References

Details

(Keywords: intl)

Attachments

(4 files)

String bundle needs to notify its callers when the cachec bundle is flushed;
otherwise, wrong UI strings or localizable resources might be used by Mozilla.
Blocks: 62177
Status: NEW → ASSIGNED
Hi, Jud:


Since you had seen this patch in bug 26291 (async loading of strres), would you
kindly review the cleaned up patch here for me?


Hi, Alec:

Since you know bundle cache better, would sr the patch?


thx
Whiteboard: patch available for review
this looks fine, but there's more to this bug, right? The string bundles
themselves need to be observers on this notification.
sr=alecf on what you've attached so far.
Why is observer notification optional (bool notify)? It would seem to me that
you should always notify.

I'm cc'ing Conrad as he's whacking runtime profile switching together (which
probably requires string bundle caches to be flushed).

What is a "string bundle" exactly? Is it an in memory
parsing/conversion/localized representation of a .properties file. .properties
files are per user right? If so, we'll definatelyl need to "flush" these when
the profile switches.
 
Hi, Alec:

One thing not obvious to me is that do we delete the bundles when the cached is
flushed? If so, after the flushing, callers probably should re-create the
bundle, isn't it?
Whiteboard: patch available for review → sr=alecf
Jud wrote:
>Why is observer notification optional (bool notify)? It would seem to me
> that you should always notify.

It's a bit tricky here. Originally, this api is called when:
1. memory low -> flush all cached bundles to reduce memory consumption.
2. profile switching (as you mentioned below) -> user locale is a profile-bound
   resource and might be different by profiles.

In case (1), you probably want to notify so the caller is aware that the bundle
handle might not be valid anymore.

In case (2), libpref is the caller since localized prefs are retrieved via
strres. However, libpref automatically reload prefs in loading a new profile.
If we notify, libpref will try to reload the bundle again... sort of wasteful.


>  I'm cc'ing Conrad as he's whacking runtime profile switching together
> (which  probably requires string bundle caches to be flushed).

Depending on which application you are working on, yes if localization resources
are profile-bound.

> What is a "string bundle" exactly? Is it an in memory
> parsing/conversion/localized representation of a .properties file. .properties
> files are per user right? If so, we'll definatelyl need to "flush" these when
> the profile switches.

We store string resources as (key, value) pairs in external *.properties
(Java property file format). The caller retrieves them via nsIStringBundle which
loads the strings from property files and store them in in-memory hash table.

> In case (2), libpref is the caller since localized prefs are retrieved via
> strres. However, libpref automatically reload prefs in loading a new profile.
> If we notify, libpref will try to reload the bundle again... sort of wasteful.

This is true, but from this:

> We store string resources as (key, value) pairs in external *.properties
> (Java property file format). The caller retrieves them via nsIStringBundle
> which loads the strings from property files and store them in in-memory hash
> table.

Aren't these being used to hold more than localized prefs? What about XUL?

hmmm. sounds like a design flaw then. You're compensating for a problem in
semantics of when the notification is fired.

thinking this through....

You want bundle users to re-load their bundles when we run low on mem, or a
profile gets changed. Each of those events (noticed via the string bundle
listening to pertinent observer topics) causes your cache to dissolve.

What exactly is the bundle user caching? a nsIStringBundle pointer? Are they
supposed to release that ptr and create a new one after receiving the
notification? If that's the case, shouldn't we hide the cache change behind the
interface? I mean, if I get a string bundle, I'd prefer not to have to know
anything about profiles or low mem conditions. I'd rather the string bundle impl
just "do the right thing" under the covers for me. Am I missing something here?

Sorry to hold up the train :-/.
>> We store string resources as (key, value) pairs in external *.properties
>> (Java property file format). The caller retrieves them via nsIStringBundle
>> which loads the strings from property files and store them in in-memory hash
>> table.
>
>Aren't these being used to hold more than localized prefs?

You got it. Localizable resources are part of customizable resources. Mozilla
use string bundle to store resources to be determined at runtime so they can
customized and extended easily.


>What about XUL?
>>What about XUL?

What are you referring to? Customizable resources are stored in external DTD
subset (*.dtd). They need to be reloaded in switching locale (or profile;
depending on what you need...)

Note that switching locale on the fly could be very complicate for full-blown
XUL application.
Jud is entirely correct (beat me to it) - we're should not be destroying old
string bundles, we should reload new .properties files. This is why I think that
all existing string bundles (independant of the cache!) should be notified that
the locale has changed.

Here's what I was envisioning for this:
- when a string bundle is created, the string bundle service adds a weak
reference to the string bundle itself to a list of created string bundles.
- when the locale changes, we go through the list and notify string bundles (via
the nsIObserver interface) that are still around that they should flush
themselves and reload..
- the cache still holds a strong reference to cached string bundles - whether or
not the cache itself should be released is up for debate.

I think that the flushing of the cache itself should be an entirely seperate API
call... i.e. in the two scenarios tao is suggesting, one of them is for
releasing unnecessary memory, and another is for flushing strings. This isn't
really a notify vs. don't notify situation. This is more like changing locale
vs. low memory, and I don't think that consitutes a boolean.

Anyway, I think we should leave the flushing of the cache up to the memory
observer mechanism, and not try to do extra work through flushBundles().
I totally agree w/ alec's last stmnt. I'd leave the mem cache flushing up to the
current low mem topic semantics. and when do local's change? if they change when
the user changes them via some menu item, and when the profile changes, the
cache flushing/bundle reloading logic should be respond to both the
profile-changed topic, as well as the locale-changed-via-UI topic.
Jud said:

>You want bundle users to re-load their bundles when we run low on mem,

Not in this case.

When memory is low, we flush the cached bundle to reduce the
memory usage, however, we DON'T want the caller to reload right away since
the memory usage would go up again. The caller reload the bundle only when they
need it.

> or a
>profile gets changed. Each of those events (noticed via the string bundle
> listening to pertinent observer topics) causes your cache to dissolve.


When memery is low, we want to release cached bundle (table) but not reload the
bundle!

I agree that the should be two apis: one for memory low; the other for locale
changes. Weak reference seems to be the solution but then we might need to change
existing usage model. Let me look into this a bit. I might not be able to finish
it thought since there are other high priority tasks.

Let me make sure this is what we want:

     When cached bundle is flushed, we always send notification so the users of
     stringbundle can take proper action. In the case of memory low, we
     don't want to reload the bundles right away so memory usage won't go up
     again. In locale switching, it is also upto the callers to decide whether
     reload or not since all chromes need to be torn up and rebuilt. Bundles
     needed in rebuilding the chromes will be (re) loaded by the callers during
     the process of chrome reconstruction.

     Conclusion: always notify by topics; never auto reload behind the scene.
 
well, I think the current behavior for the memory-low condition is good
enough... it "flushes" the cache held by the SERVICE by releasing the strong
references to the string bundles - the bundles which are not in use will go away
entirely. I already implemented that part 3-4 months ago (see
nsStringBundleService::Observe)


What this bug is supposed to be for (or so I thought) was the situation where
we're flushing all of the string BUNDLES themselves because the string
bundles(i.e. nothing to do with the cache) to tell them that their properties
files are invalid...

once you fix this second part (i.e. this bug) then maybe we should consider
having the memory pressure observer ALSO flush the string bundles themselves..
>What this bug is supposed to be for (or so I thought) was the situation where
>we're flushing all of the string BUNDLES themselves because the string
>bundles(i.e. nothing to do with the cache) to tell them that their properties
>files are invalid...

It seems to me a reasonable way to address this is
  1. flush bundle cache first
  2. notify all bundle users to release bundles and then re-create (load)
     the bundles. Bundle users need to be observers of the locale-changed topic.

Sounds good?

>once you fix this second part (i.e. this bug) then maybe we should consider
>having the memory pressure observer ALSO flush the string bundles themselves..

right. this should be done outside of stringbundle.

I am taking out "sr=alecf"
Whiteboard: sr=alecf
here's a patch to make the string bundles flush their data when they get the
memory-pressure observer notification.
oops, I attached the patch to bug 62748
jud sez r=valeski
adding erik for super review

Hi, Alec:

Thanks for the good work. Some minor points.

  1. you don't need nsILocale in creating a new bundle. The arg. is being
     obsolete.
  2. I notice that you call OpenInputStream directly to reload the stream. Are
     we opting out the async loading model? Thanks to valeski & danm, I manage
     to get the async loading working. I suspect we'll need to revamp the
     stream loading and bundling part in your patch.
  3. In GetStringFromXXX(), you call LoadBundle() to assure that the bundle is
     loaded. However, will we run into racing condition when mProps==nsull and
     GetStringFromXX() is called from vairous places/threads?

BTW, Erik is out till Jan,'01.
well, all I did was move existing code, I didn't change the way it actually works.

You're right about a race condition, I'll add an autolock and try again
Keywords: intl
Keywords: nsbeta1
Hi, Alecf:

I had checked in my patch for 26291. Please revise your patch for the
weakreference part.

Reassign to alecf for the weak-reference patch.
Assignee: tao → alecf
Status: ASSIGNED → NEW
Changed QA contact to tao@netscape.com.
QA Contact: teruko → tao
Status: NEW → ASSIGNED
Target Milestone: --- → mozilla1.0
Marking nsbeta1+, mozilla0.9
Keywords: nsbeta1nsbeta1+
Target Milestone: mozilla1.0 → mozilla0.9
we can survive 0.9 without this, but don't worry I'll get to it for nsbeta1
we can survive 0.9 without this, but dont' worry I'll get it for nsbeta1
Target Milestone: mozilla0.9 → mozilla0.9.1
as discussed in team meeting, moving all Nav+ team members nsbeta1+ P3 bugs from 
mozilla0.9.1 to mozilla0.9.2. 
Target Milestone: mozilla0.9.1 → mozilla0.9.2
Priority: P3 → P2
Target Milestone: mozilla0.9.2 → mozilla0.9.1
Whiteboard: fix in hand
Whiteboard: fix in hand
nav triage team:

So is this REALLY critical for beta? We're itchin' to minus this one.
Alec, it seems in rather bad form to have moved the target milestone on 4/20
without even a comment about why you had undone the triage team's decision from
that same day.  Is this a common practice?  Is it not possible to discuss this
type of issue with the triage team rather than follow behind like that?

I don't understand why we would hold shipment of the beta for this, please explain.
nav triage: removing nsbeta1+, changing to nsbeta1- and moving to mozilla0.9.2 
at least. Not clear why this holds shipment of beta or rtm. 
Keywords: nsbeta1+nsbeta1-
Target Milestone: mozilla0.9.1 → mozilla0.9.2
this holds RTM because when we switch locales, any cached string bundles still
hold onto the strings from the previous locale.
Agreed, we should fix this for M0.9.2.
Vishy - This sounds to me, like we might have mixed data in the UI, right? Pls 
explain . . . If this is the case, this would be a stop ship (data loss) for me.

Removing minus until we talk.
Keywords: nsbeta1-nsbeta1, nsCatFood
Per Vishy's request, setting priority P1, so that we can triage with Nav for 
M0.9.2.
Keywords: rtm
Priority: P2 → P1
To keep this bug on the mozilla0.9.2 list, we need an explanation of the user 
impact - i.e. a sequence of user steps that leads to bad things happening. 
Otherwise its an architectural work that needs to move to mozilla1.0. I'll keep 
this bug on the m0.9.2 radar till 5/30, if we dont get a reason for it to be on 
the rtm stopping radar, then we'll defer it. 
the problem is that if you switch locales at runtime, your UI will contain a mix
of the previous and current languages... i.e. if you switch from English to
Japanese, your UI will contain a mix of English and Japanese.
we shd check this in early in the m0.9.2 cycle as it seems like it could have
side effects. 
IQA (teruko), could you assess what the behaviour of the app is from the user 
perspective when a locale switch is done? we need to decide whether to fix this, 
or just require a restart. 
QA Contact: tao → jonrubin
Whiteboard: 2 days, eta 6/15
an update: I have a rough cut of this thing almost ready for consumption..
Whiteboard: 2 days, eta 6/15 → 1 more day, eta 6/18
Whiteboard: 1 more day, eta 6/18 → 1 more day, eta 6/19
Alec, how big is this change? Is there any risk of taking this at this stage? We 
may be able to live with requiring a restart when the locale is switched rather 
than taking this, if it is a large patch and needs extensive testing. 
the change is large, but the code that I'm adding is only run when locale is
switched. If this is the only thing holding back locale switching then I think
we should take this bug - I am very close, and expect to finish as soon as I
finish bug 73141.

If there are other bugs holding back locale switching, then we should not bother
with this bug.
Whiteboard: 1 more day, eta 6/19 → need new eta.
Ray/Frank/Jaime - I'm not sure who can answer Alec's last question - but can one
of you figure it out? Its important to know this soon. thanks, Vishy
ok the above patch:
- keeps a weak reference to ever string bundle it has ever created. 
- Whenever flushBundles() is called (such as by the chrome registry) then the
list of weak bundles is walked, removing bundles that have gone away, and
flushing those that have
- it also removes createAsyncBundle, which has never been used, and was
basically broken anyway (long story)

Looking for r=/sr=
Whiteboard: need new eta. → fix in hand, need r=, sr=, a=
Keywords: nsBranch
Alec, is the change needed for run time locale switching only?
Jaime, do we plan to support run time locale switch?
yes, this is for runtime profile switching
er, locale switching
r=nhotta

FlushBundles() is also called by nsProfile.cpp and it seems to be called
everytime at app startup. Does this mean currently the cache is not flushed
correctly at the profile code without this fix?


yes, that is true, but the only string bundles that are likely loaded are the
profile manager's bundles, and that is only true if the profile manager UI appears.

The call to FlushBundles in nsProfile.cpp is probably done so that, if the
default locale (and that used by the profile mgr UI) is X and the user selects a
profile that is locale Y, the string bundles from locale X are flushed. This
call to FlushBundles pre-dates profile change notifications. It would probably
be better if the string bundle code was a profile change observer. Then it could
probably be smarter about if and when it needed to flush.
Nhotta - Not sure I follow this conversation, "Jaime, do we plan to support run 
time locale switch?" Can you pls come by and explain this one to me.
Run time locale switch allows the user to switch locale without requiring
restart the application.
I'm going to check this into the trunk today but I need an sr=. Asking blizzard
since there's no i18n-oriented sr=
Whiteboard: fix in hand, need r=, sr=, a= → fix in hand, need sr=, a=
pushing out. 0.9.2 is done. (querying for this string will get you the list of
the 0.9.2 bugs I moved to 0.9.3)
Target Milestone: mozilla0.9.2 → mozilla0.9.3
sr=blizzard
Whiteboard: fix in hand, need sr=, a= → fix in hand, need a=
ok, fix is checked into trunk. cross your fingers!
Keywords: vtrunk
I'm crashing in nsStringBundleService::flushNotificationList on today's build 
immediately after I press the "Start Mozilla" button in the "Select User 
Profile" dialog.

Here's the stack trace:

nsCOMPtr<nsIWeakReference>::assign_with_AddRef(nsISupports * 0x0338cb20) line 
962 + 9 bytes
nsCOMPtr<nsIWeakReference>::operator=(nsIWeakReference * 0x0338cb20) line 584
nsStringBundleService::flushNotificationList() line 839
nsStringBundleService::FlushBundles(nsStringBundleService * const 0x02c1df70) 
line 864
nsProfile::SetCurrentProfile(nsProfile * const 0x00a3e5d0, const unsigned short 
* 0x033bef90) line 1030 + 32 bytes
nsProfile::StartApprunner(nsProfile * const 0x00a3e5d0, const unsigned short * 
0x033bef90) line 1726 + 16 bytes
XPTC_InvokeByIndex(nsISupports * 0x00a3e5d0, unsigned int 0x00000010, unsigned 
int 0x00000001, nsXPTCVariant * 0x0012cd8c) line 139
XPCWrappedNative::CallMethod(XPCCallContext & {...}, XPCWrappedNative::CallMode 
CALL_METHOD) line 1881 + 42 bytes
XPC_WN_CallMethod(JSContext * 0x00f1a1b0, JSObject * 0x00d5c860, unsigned int 
0x00000001, long * 0x02dfd788, long * 0x0012cfc0) line 1252 + 11 bytes
js_Invoke(JSContext * 0x00f1a1b0, unsigned int 0x00000001, unsigned int 
0x00000000) line 807 + 23 bytes
js_Interpret(JSContext * 0x00f1a1b0, long * 0x0012dd64) line 2702 + 15 bytes
js_Invoke(JSContext * 0x00f1a1b0, unsigned int 0x00000001, unsigned int 
0x00000002) line 824 + 13 bytes
js_InternalInvoke(JSContext * 0x00f1a1b0, JSObject * 0x02de4da8, long 
0x02de5ad0, unsigned int 0x00000000, unsigned int 0x00000001, long * 
0x0012df3c, long * 0x0012de8c) line 896 + 20 bytes
JS_CallFunctionValue(JSContext * 0x00f1a1b0, JSObject * 0x02de4da8, long 
0x02de5ad0, unsigned int 0x00000001, long * 0x0012df3c, long * 0x0012de8c) line 
3320 + 31 bytes
nsJSContext::CallEventHandler(nsJSContext * const 0x00f19100, void * 
0x02de4da8, void * 0x02de5ad0, unsigned int 0x00000001, void * 0x0012df3c, int 
* 0x0012df38, int 0x00000000) line 941 + 33 bytes
nsJSEventListener::HandleEvent(nsJSEventListener * const 0x03227700, 
nsIDOMEvent * 0x033b7de4) line 139 + 57 bytes
nsEventListenerManager::HandleEventSubType(nsListenerStruct * 0x03227680, 
nsIDOMEvent * 0x033b7de4, nsIDOMEventTarget * 0x03225a98, unsigned int 
0x00000008, unsigned int 0x00000007) line 1161 + 20 bytes
nsEventListenerManager::HandleEvent(nsEventListenerManager * const 0x03227750, 
nsIPresContext * 0x02be6a30, nsEvent * 0x0012ea44, nsIDOMEvent * * 0x0012e908, 
nsIDOMEventTarget * 0x03225a98, unsigned int 0x00000007, nsEventStatus * 
0x0012ea8c) line 2131 + 36 bytes
nsXULElement::HandleDOMEvent(nsXULElement * const 0x03225a90, nsIPresContext * 
0x02be6a30, nsEvent * 0x0012ea44, nsIDOMEvent * * 0x0012e908, unsigned int 
0x00000001, nsEventStatus * 0x0012ea8c) line 3635
PresShell::HandleDOMEventWithTarget(PresShell * const 0x02c1d200, nsIContent * 
0x03225a90, nsEvent * 0x0012ea44, nsEventStatus * 0x0012ea8c) line 5677 + 39 
bytes
nsButtonBoxFrame::MouseClicked(nsIPresContext * 0x02be6a30, nsGUIEvent * 
0x0012ec40) line 181
nsButtonBoxFrame::HandleEvent(nsButtonBoxFrame * const 0x02de8d10, 
nsIPresContext * 0x02be6a30, nsGUIEvent * 0x0012ec40, nsEventStatus * 
0x0012ef4c) line 128
PresShell::HandleEventInternal(nsEvent * 0x0012ec40, nsIView * 0x00000000, 
unsigned int 0x00000001, nsEventStatus * 0x0012ef4c) line 5645 + 41 bytes
PresShell::HandleEventWithTarget(PresShell * const 0x02c1d200, nsEvent * 
0x0012ec40, nsIFrame * 0x02de8d10, nsIContent * 0x03225a90, unsigned int 
0x00000001, nsEventStatus * 0x0012ef4c) line 5603 + 22 bytes
nsEventStateManager::CheckForAndDispatchClick(nsEventStateManager * const 
0x03240810, nsIPresContext * 0x02be6a30, nsMouseEvent * 0x0012f058, 
nsEventStatus * 0x0012ef4c) line 2455 + 61 bytes
nsEventStateManager::PostHandleEvent(nsEventStateManager * const 0x03240818, 
nsIPresContext * 0x02be6a30, nsEvent * 0x0012f058, nsIFrame * 0x02de8d10, 
nsEventStatus * 0x0012ef4c, nsIView * 0x02be83d0) line 1540 + 28 bytes
PresShell::HandleEventInternal(nsEvent * 0x0012f058, nsIView * 0x02be83d0, 
unsigned int 0x00000001, nsEventStatus * 0x0012ef4c) line 5650 + 43 bytes
PresShell::HandleEvent(PresShell * const 0x02c1d204, nsIView * 0x02be83d0, 
nsGUIEvent * 0x0012f058, nsEventStatus * 0x0012ef4c, int 0x00000001, int & 
0x00000001) line 5557 + 25 bytes
nsView::HandleEvent(nsView * const 0x02be83d0, nsGUIEvent * 0x0012f058, 
unsigned int 0x0000001c, nsEventStatus * 0x0012ef4c, int 0x00000001, int & 
0x00000001) line 377
nsViewManager::DispatchEvent(nsViewManager * const 0x02be89f0, nsGUIEvent * 
0x0012f058, nsEventStatus * 0x0012ef4c) line 2056
HandleEvent(nsGUIEvent * 0x0012f058) line 68
nsWindow::DispatchEvent(nsWindow * const 0x02be81c4, nsGUIEvent * 0x0012f058, 
nsEventStatus & nsEventStatus_eIgnore) line 720 + 10 bytes
nsWindow::DispatchWindowEvent(nsGUIEvent * 0x0012f058) line 741
nsWindow::DispatchMouseEvent(unsigned int 0x0000012d, nsPoint * 0x00000000) 
line 4240 + 21 bytes
ChildWindow::DispatchMouseEvent(unsigned int 0x0000012d, nsPoint * 0x00000000) 
line 4489
nsWindow::ProcessMessage(unsigned int 0x00000202, unsigned int 0x00000000, long 
0x0104013a, long * 0x0012f498) line 3203 + 24 bytes
nsWindow::WindowProc(HWND__ * 0x0077053e, unsigned int 0x00000202, unsigned int 
0x00000000, long 0x0104013a) line 988 + 27 bytes
USER32! 77f013ed()
JS3250! js_FunctionClass + 39170 bytes
backed myself out.. I don't know what's up with this, but doing unix testing
now..
Keywords: vtrunk
how did the unix testing go on this? 
Keywords: rtmnsrtm
dropping nsBranch keyword. 
Keywords: nsBranch
moving to mozilla1.0 at this point. 
Target Milestone: mozilla0.9.3 → mozilla1.0
If we have a fix, and this bug has been given a nsbeta1+, and nsCatFood+, then
why wouldn't we slot this one into M0.9.4?
It was backed out due to Simon Montagu's finding on 2001-07-03 05:17.

ah! </8^(
mass change, switching qa contact from jonrubin to ruixu.
QA Contact: jonrubin → ruixu
Bugs targeted at mozilla1.0 without the mozilla1.0 keyword moved to mozilla1.0.1 
(you can query for this string to delete spam or retrieve the list of bugs I've 
moved)
Target Milestone: mozilla1.0 → mozilla1.0.1
Target Milestone: mozilla1.0.1 → mozilla1.2alpha
Whiteboard: fix in hand, need a=
pushing a few bugs out to 1.2beta - too much stuff on my 1.2alpha plate!
Priority: P1 → P2
Target Milestone: mozilla1.2alpha → mozilla1.2beta
mozilla 1.1alpha is more or less done, so I'm moving non-critical mozilla1.2beta
bugs out to the next milestone to make room for the mozilla1.1alpha bugs that
didn't make it.
Target Milestone: mozilla1.2beta → mozilla1.3alpha
Furturing bugs that keep getting knocked from milestone to milestone...if you
feel this is in error, please nominate this bug using the appropriate keyword.
Target Milestone: mozilla1.3alpha → Future
QA Contact: ruixu → ylong
patch was applied but backed out.  Is this approach a dead end?
Assignee: alecf → smontagu
Status: ASSIGNED → NEW
QA Contact: amyy → i18n
Moving to p3 because no activity for at least 1 year(s).
See https://github.com/mozilla/bug-handling/blob/master/policy/triage-bugzilla.md#how-do-you-triage for more information
Priority: P2 → P3

We're not going to invest in StringBundle anymore, and LocaleService now reports language changes.

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: