Closed Bug 140698 Opened 22 years ago Closed 22 years ago

mozilla(gtk2) coredump on Solaris

Categories

(Core :: XUL, defect)

Product:
Core
Component:
XUL
Platform:
Sun
Solaris
Type:
defect
Priority:
Not set
Severity:
blocker

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: jay.yan, Assigned: yinbolian)

References

Details

Attachments

(2 files, 1 obsolete file)

debug information of mozilla(gtk2.0.2) on Solaris
8.07 KB, text/plain
Details
Patch as Owen suggested
1.21 KB, patch
blizzard
: review+
Details | Diff | Splinter Review 
John, Bruce, Bolian, and I build mozilla using gtk2.0.0 on Solaris, building
process runs smoothly. after we start up mozilla, coredump happens.

Compling Enviroment:
  Solaris8 and Solari9
  gtk2.0.0(including its dependent libraries, shiping with Gnome Beta3) 
  CC (Ship with Forte6 update 2)

Both Bolian and I tried to use DBX to debug it. it is the debug info output by
DBX: (DBX is debugger of Forte)
 
***********below is output of DBX*************************************
Reading mozilla-bin
Reading ld.so.1
Reading libgkgfx.so
Reading libjsj.so
Reading libmozjs.so
Reading libxpcom.so
Reading libplds4.so
Reading libplc4.so
Reading libnspr4.so
Reading libpthread.so.1
Reading libdl.so.1
Reading librt.so.1
Reading libsocket.so.1
Reading libm.so.1
Reading libCrun.so.1
Reading libw.so.1
Reading libthread.so.1
Reading libc.so.1
Reading libnsl.so.1
Reading libaio.so.1
Reading libmd5.so.1
Reading libmp.so.2
Reading libc_psr.so.1
detected a multithreaded program
(/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) run
Running: mozilla-bin 
(process id 5278)
Reading libdemangle.so.1
File descriptors set to 512
Type Manifest File:
/export/admin/build/mozilla1.0-CC/mozilla/dist/bin/components/xpti.dat
nsNativeComponentLoader: autoregistering begins.
nsNativeComponentLoader: autoregistering succeeded
Reading libjsloader.so
nNCL: registering deferred (0)
Reading libembedcomponents.so
Reading libcaps.so
Reading libmozz.so
Reading libxpconnect.so
Reading libpref.so
Reading libnecko.so
Reading libnsappshell.so
Reading libwidget_gtk2.so
Reading libX11.so.4
Reading libgtk-x11-2.0.so.0.0.1
Reading libgdk-x11-2.0.so.0.0.1
Reading libatk-1.0.so.0.0.1
Reading libgdk_pixbuf-2.0.so.0.0.1
Reading libpangox-1.0.so.0.0.1
Reading libpango-1.0.so.0.0.1
Reading libgobject-2.0.so.0.0.1
Reading libgmodule-2.0.so.0.0.1
Reading libglib-2.0.so.0.0.1
Reading libXext.so.0
Reading libintl.so.1.0.1
Reading librdf.so
Reading libprofile.so
Reading libuconv.so
Reading libunicharutil.so
warning: property cp1256 already exists
Reading libucvlatin.so
Reading libchrome.so
Reading libhtmlpars.so
Reading liburiloader.so
Reading libgfx_gtk.so
Reading libXp.so.1
GFX: dpi=90 t2p=0.0625 p2t=16 depth=24
Reading UTF-8%8859-1.so
Reading libdocshell.so
WEBSHELL+ = 1
Reading libjsdom.so
Reading libgkcontent.so
Reading libnslocale.so
Reading liboji.so
Reading libgkplugin.so
Reading libmozbrwsr.so
Reading libappcomps.so
Reading libstrres.so
Reading libimglib2.so
Reading libjar50.so
warning: property switchskins already exists
warning: property switchskinstitle already exists
Reading libcookie.so
WEBSHELL+ = 2
Reading libchardet.so
Reading libgklayout.so
t@1 (l@1) signal SEGV (no mapping at the fault address) in
gtk_tooltips_force_window dbx: warning: can't find file
"/gnome/2.0/community/src-2.0_06/gtk-multihead/gtk/gtktooltips.lo"
dbx: warning: see `help finding-files'
at 0xfe2c0f60
0xfe2c0f60: gtk_tooltips_force_window+0x00f0:   ld      [%l0 + 0x4], %l0
Current function is nsLookAndFeel::InitColors (optimized)
  424     gtk_tooltips_force_window(tooltips);
(/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) 

********************above is output of DBX*******************************

  

it is the function stack :

*************below is function stack when segment fault happens*************

  424     gtk_tooltips_force_window(tooltips);
(/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) where
current thread: t@1
  [1] gtk_tooltips_force_window(0x1f99c8, 0xb9340, 0x1, 0x0, 0x0, 0x0), at
0xfe2c0f60
=>[2] nsLookAndFeel::InitColors() (optimized), at 0xfe640308 (line ~424) in
"nsLookAndFeel.cpp"
  [3] nsLookAndFeel::nsLookAndFeel(this = ???) (optimized), at 0xfe63f2f8 (line
~69) in "nsLookAndFeel.cpp"
  [4] nsLookAndFeelConstructor(aOuter = ???, aIID = STRUCT, aResult = ???)
(optimized), at 0xfe63d3e0 (line ~50) in "nsWidgetFactory.cpp"
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/xpcom/build/nsGenericFactory.o"
  [5] nsGenericFactory::CreateInstance(0x1f8038, 0x0, 0xfc5bf228, 0xffbfe6a8,
0x2c48, 0xff1d1078), at 0xff152920
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/xpcom/build/nsComponentManager.o"
  [6] nsComponentManagerImpl::CreateInstance(0x400c8, 0xfc5d4968, 0x0,
0xfc5bf228, 0xffbfe6a8, 0x10a4), at 0xff0db064
  [7] nsComponentManagerImpl::GetService(0x400c8, 0xfc5d4968, 0xfc5bf228,
0xffbfe7b4, 0xda97c, 0xa22176e1), at 0xff0db5d4
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/xpcom/build/nsComponentManagerUtils.o"
  [8] nsGetServiceByCID::operator()(0xffbfe87c, 0xfc5bf228, 0xffbfe7b4,
0xff155184, 0x0, 0x0), at 0xff155228
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/xpcom/build/nsCOMPtr.o"
  [9] nsCOMPtr_base::assign_from_helper(0x1f8f38, 0xffbfe87c, 0xfc5bf228,
0xffbfe88c, 0x0, 0x0), at 0xff151c7c
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/layout/build/nsPresContext.o"
  [10] nsCOMPtr<nsILookAndFeel>::operator=(0x1f8f38, 0xffbfe87c, 0xffbfe88c,
0x207bb8, 0x0, 0x0), at 0xfc39f6e4
  [11] nsPresContext::GetLookAndFeel(0x1f8ef8, 0xffbfe91c, 0x0, 0xfc5d49f0,
0xfc3a2d44, 0x0), at 0xfc39c840
  [12] nsPresContext::nsPresContext(0x1f8ef8, 0xffffffff, 0x1, 0x1f914c,
0x1f90b4, 0x1f9090), at 0xfc399a84
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/layout/build/nsGalleyContext.o"
  [13] GalleyContext::GalleyContext(0x1f8ef8, 0x0, 0x0, 0x0, 0x0, 0x0), at
0xfc395108
  [14] NS_NewGalleyContext(0xffbfea44, 0xffbfea34, 0x0, 0x39670, 0xffbfeb0c,
0xff0e3f0c), at 0xfc395414
  [15] CreateNewGalleyContext(aOuter = ???, aIID = STRUCT, aResult = ???)
(optimized), at 0xfc13f730 (line ~218) in "nsLayoutModule.cpp"
  [16] nsGenericFactory::CreateInstance(0x1f7fb8, 0x0, 0xfd445f0c, 0xffbfebec,
0xff1c076c, 0xff1d1078), at 0xff152920
  [17] nsComponentManagerImpl::CreateInstance(0x400c8, 0xfd46d8b4, 0x0,
0xfd445f0c, 0xffbfebec, 0x10a4), at 0xff0db064
  [18] nsCreateInstanceByCID::operator()(0xffbfed54, 0xfd445f0c, 0xffbfebec,
0xff154f7c, 0xfebc087c, 0xffbfecd4), at 0xff154fe8
  [19] nsCOMPtr_base::assign_from_helper(0x1a30a0, 0xffbfed54, 0xfd445f0c,
0xffbfed64, 0xfcc2e300, 0x2), at 0xff151c7c
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/content/build/nsDocumentViewer.o"
  [20] nsCOMPtr<nsIPresContext>::operator=(0x1a30a0, 0xffbfed54, 0xffbfed64,
0xfd7f7e3c, 0xff358e04, 0xa000), at 0xfd12ebd4
  [21] DocumentViewerImpl::InitInternal(0x1a3068, 0xfd7f7e3c, 0x1d77f8,
0xff357d90, 0x1a30a0, 0xfd42a3e8), at 0xfd11417c
  [22] DocumentViewerImpl::Init(0x1a3068, 0x10c548, 0x1d77f8, 0xffbfee64,
0xfd46db98, 0xfd617898), at 0xfd114004
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/docshell/build/nsDocShell.o"
  [23] nsDocShell::SetupNewViewer(0x11fa30, 0x0, 0xffbfeeac, 0xfd70b804,
0x11fac0, 0x0), at 0xfd6a45cc
  [24] nsDocShell::Embed(0x11fa30, 0x1a3068, 0x11fb34, 0x0, 0x0, 0x0), at
0xfd6a2164
  [25] nsDocShell::CreateContentViewer(0x11fa30, 0xffbff048, 0x120158,
0xffbff04c, 0xffbff05c, 0xffbff060), at 0xfd6a34ac
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/docshell/build/nsDSURIContentListener.o"
  [26] nsDSURIContentListener::DoContent(0x7ed60, 0xffbff1d4, 0x0, 0x120158,
0xffbff258, 0xffbff214), at 0xfd6cd958
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/uriloader/build/nsURILoader.o"
  [27] nsDocumentOpenInfo::DispatchContent(0xbfee8, 0x120158, 0x7ed60, 0x11c1c0,
0x11fa30, 0x14), at 0xfd81f028
  [28] nsDocumentOpenInfo::OnStartRequest(0xbfee8, 0x120158, 0x0, 0x0, 0x4e420,
0x0), at 0xfd81eaa8
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/netwerk/build/nsInputStreamChannel.o"
  [29] nsStreamIOChannel::OnStartRequest(0x120158, 0x1201dc, 0x0, 0xfe6f01d8,
0x11c460, 0x0), at 0xfe6edddc
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/netwerk/build/nsRequestObserverProxy.o"
  [30] nsOnStartRequestEvent::HandleEvent(0x120018, 0x15ceec, 0x434, 0x2800,
0xfe865228, 0x29fc), at 0xfe708418
  [31] nsARequestObserverEvent::HandlePLEvent(0x12001c, 0x1, 0x1, 0x12001c,
0xd0a5c, 0xff1c34e4), at 0xfe70698c
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/xpcom/build/plevent.o"
  [32] PL_HandleEvent(0x12001c, 0x0, 0x1000, 0x137c, 0x1000, 0x1380), at
0xff0efd5c
  [33] PL_ProcessPendingEvents(0x688c0, 0x1, 0xd0c94, 0x0, 0xff1c076c,
0x63680024), at 0xff0efb9c
dbx: warning: can't find file
"/export/admin/build/mozilla1.0-CC/mozilla/xpcom/build/nsEventQueue.o"
  [34] nsEventQueueImpl::ProcessPendingEvents(0x68890, 0x0, 0x2cec0, 0x0,
0xfff71470, 0x0), at 0xff0f1e98
  [35] event_processor_callback(source = ???, condition = ???, data = ???)
(optimized), at 0xfe63bf14 (line ~57) in "nsAppShell.cpp"
dbx: warning: can't find file
"/gnome/2.0/community/src-2.0_06/glib/glib/giounix.lo"
  [36] g_io_unix_dispatch(0x1a28e8, 0xfe63befc, 0x68890, 0xffbff6e0, 0xfdc893c4,
0x1), at 0xfdd2a184
dbx: warning: can't find file
"/gnome/2.0/community/src-2.0_06/glib/glib/gmain.lo"
  [37] g_main_dispatch(0xb9120, 0x1a28e8, 0xffffffff, 0xfdcad72c, 0xfdc8c314,
0x1), at 0xfdce0370
  [38] g_main_context_dispatch(0xb9120, 0x7fffffff, 0x1a57c0, 0x2, 0x2, 0x2), at
0xfdce2328
  [39] g_main_context_iterate(0xb9120, 0x1, 0x1, 0x1a44e0, 0xfe00ef98, 0x2), at
0xfdce2a88
  [40] g_main_loop_run(0x1a57a8, 0x1a57a8, 0x0, 0xfe6710f0, 0x230, 0x1a28a0), at
0xfdce3ab0
dbx: warning: can't find file
"/gnome/2.0/community/src-2.0_06/gtk-multihead/gtk/gtkmain.lo"
  [41] gtk_main(0x68890, 0xfe600594, 0xfe6586b9, 0xffbff9a4, 0x31b34,
0xfe66de68), at 0xfe18eaf0
  [42] nsAppShell::Run(this = ???) (optimized), at 0xfe63c300 (line ~111) in
"nsAppShell.cpp"
  [43] nsAppShellService::Run(this = ???) (optimized), at 0xfe5af6a8 (line ~308)
in "nsAppShellService.cpp"
  [44] main1(argc = ???, argv = ???, nativeApp = ???) (optimized), at 0x17c0c
(line ~1432) in "nsAppRunner.cpp"
  [45] main(argc = ???, argv = ???) (optimized), at 0x18314 (line ~1775) in
"nsAppRunner.cpp"
(/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) 

***************above is function stack when segment fault happens*************
Some comments:

1 It seems that the problem exists on Mozilla's code, not on gtk2's code or
compling building enviroment, I am not 100% sure now, let me go deeper and
see......

2 Althought I do not think it is becasue of complier, I will also try gcc2.95.3
to build mozilla(gtk2) on Solaris and see what will happen.
Blocks: gtk2
I am not sure that this is not becasue Compiler.

I located why the coredump happened(I comment several lines of code,
mozilla(gtk2) can run on Solaris), please see:

nsLookAndFeel::InitColors()
{
  sColorsInitialized = PR_TRUE;
  GtkStyle *style;
  
  // tooltip foreground and background
/****** delete following lines **************************
  GtkTooltips *tooltips = gtk_tooltips_new();
  gtk_object_ref(GTK_OBJECT(tooltips));
  gtk_object_sink(GTK_OBJECT(tooltips));
  gtk_tooltips_force_window(tooltips);
  GtkWidget *tip_window = tooltips->tip_window;
  gtk_widget_set_rc_style(tip_window);

  style = gtk_widget_get_style(tip_window);
  sInfoBackground = GDK_COLOR_TO_NS_RGB(style->bg[GTK_STATE_NORMAL]);
  sInfoText = GDK_COLOR_TO_NS_RGB(style->fg[GTK_STATE_NORMAL]);

  gtk_object_unref(GTK_OBJECT(tooltips));

********delete above lines  **************************************/
  // menu foreground & menu background
  GtkWidget *accel_label = gtk_accel_label_new("M");
  GtkWidget *menuitem = gtk_menu_item_new();
  GtkWidget *menu = gtk_menu_new();
  gtk_object_ref(GTK_OBJECT(menu));
  gtk_object_sink(GTK_OBJECT(menu));
.............


I comment the code of tooltips part, then I can start mozilla on Solaris, the
compiler I used is Forte's CC.

Let us see mozilla(gtk1.2)'s accordingly part to compare:

****code snip of mozilla(gtk1.2)****
 // tooltip foreground and background
  GtkTooltips *tooltips = gtk_tooltips_new();
  gtk_tooltips_force_window(tooltips);
  GtkWidget *tip_window = tooltips->tip_window;
  gtk_widget_set_rc_style(tip_window);

  style = gtk_widget_get_style(tip_window);
  sInfoBackground = GDK_COLOR_TO_NS_RGB(style->bg[GTK_STATE_NORMAL]);
  sInfoText = GDK_COLOR_TO_NS_RGB(style->fg[GTK_STATE_NORMAL]);

  gtk_object_unref(GTK_OBJECT(tooltips));
***code snip of mozilla(gtk1.2)*****


Only two lines are added while porting gtk1.2 to gtk2: gtk_object_ref() and
gtk_object_sink(), I think it is Gtk2/GtkObject's feature.

I will go on.... but any comment is very very useful for me.


add bryner to cc list because I saw from the license notes that bryner was the
contrubitor of file nsLookandFeel.cpp
Sure looks like a reference counting bug to me, at first glance.

with respect to the tooltip object:
g_object_sink() (I believe) makes the object lose it's internal ref; that's why
it's preceded by the g_object_ref() call.  But after you access its internal
data, you unref() it.  So the tooltip object should go away at this point; maybe
the call to gtk_tooltips_force_window () is relying on the window in the tooltip
staying alive?

As an experiment, try omitting the g_object_sink() call, or the g_object_unref()
call; unsafe in the long term since it will leak horribly, but maybe useful as a
diagnostic.
I see you are using the multihead branch of gtk+ 2.0. 
Can you reproduce this problem with gtk+ 2.0 without multihead support ?
The easiest way to do that is to grab the "normal" gtk+2.0.2 from here :
ftp://www.gtk.org/pub/gtk/v2.0/gtk+-2.0.2.tar.gz.
compile and install it (you shouldn't have to recompile glib,pango nor atk).

If mozilla works with this version, then it is a bug in my code.

Also could you add a more detailed stack trace, I'd like to see exactly where
in gtk_tooltips_force_window it is crashing.

FYI 
gtk_tooltips_force_window creates the tooltip gtkwindow and associated label 
if need.

Looks like some kind of refcnting bug.  Is there a specific site where this is
happening?  Is it happening on startup?  I need some more context here.
thanks all the comments, and sorry for the delay reply because of the time zone
difference.

Chris, yes, it happens on the start( call InitColors the first time, and I
believe this function is only be called on the start phase), you can see all the
context comment #1, it recorded all what happened after I lanuch mozilla using dbx.

Bill and Erwann, good suggestion, I will try as your suggestion,

But I am afraid perhaps I can not finish it before our long holiday from May 1st
to May7th( it is a Chinese traditional hoilday) and we will move to new site
this afternoon, all the SPARC is down during this long holiday and I have no
SPARC machine at home:-(
We are back to office now.
Today I tried as Bill's suggestion, coredump still happens when
gtk_tooltips_force_window is called, so we should get more function stacks
inside gtk2+, now I am trying to compile gtk+2.0.2 on Solaris, but some
compiling errors happened, attathed file are the error statement snip.

The command lines I used are:
configure --without-PACKAGE --without-libpng --without-libtiff
--without-libtiff
make

I am using Forte's CC and /usr/ccs/bin/make.

I will try again tommorrow, before that, can I have get any suggestion(or doc)
about how to setup compiling enviroment for gtk2(glib2, pango...) on Solaris?
Becasue without that, China team can not locate where the similar errors are:
on mozilla, on gtk2 or other libraries?
thanks
Jay
Where are you getting those packages now?
as Erwann's suggestion, I got the source tarball of gtk+2.02 fron 
ftp://www.gtk.org/pub/gtk/v2.0/gtk+-2.0.2.tar.gz.
I got Forte and make from Solaris's installtion CD.
I successed in building gtk+-2.0.2 on solaris, this time I used gmake instead of
Sun's make utility:-)

and I used 
configure --prefix=/opt/gtk2.2.2 --without-png --without-tiff --without-jpeg
--enable-debug=yes
I built mozilla with gtk2.0.2. (In comment #11, I built gtk2.0.2 according to
Erwaan's suggestion)

This time the refcnt coredump does not exist any more, because libgklayout.so
is loaded sucessfully.

But other coredump happened when libgfx2.so is loaded, perhaps I need to build
glib, pango, atk and other image loaders.

Anyway, I think our assumption is right: a refcnt bug of gtk2.0.1(last time I
used gtk2.0.1 not gtk2.0.0, it is shipped with Gnome2.0 beta3:
/net/gnome-download.ireland/gnome/2.0/2.0-06) leads to mozilla's last coredump
on Solaris.

Ervaan: How should we deal with this bug?
Attachment #82802 - Attachment is obsolete: true
This is a bug in gtk+ 2.1, I've logged a bug and a patch to fix this problem.

http://bugzilla.gnome.org/show_bug.cgi?id=81651

So this bug can be closed as soon as the gtk bug is closed.


FYI, I'm only using gtk 2.0 at the moment and I don't indent to upgrade for a
while.  Is it wise for you guys to be using a CVS tip version of gtk?  I don't
think so.
*** Bug 145601 has been marked as a duplicate of this bug. ***
Status: NEW → ASSIGNED
Any code that accesses the tooltips window is doing somethign illegal. 
I can't really see the relationship of the fix in b.g.o. 81651 to this bug, 
but there are certainly  plenty of easy ways to get the colors.

The cleanest way to get your colors is probably:

            style = gtk_rc_get_style_by_paths (gtk_settings_get_default(),
				     "gtk-tooltips", "GtkWindow",
				     GTK_TYPE_WINDOW);
  
  g_print ("Tooltips fg color is %#4x, %#4x, %#4x\n",
	   style->bg[GTK_STATE_NORMAL].red,
	   style->bg[GTK_STATE_NORMAL].green,
	   style->bg[GTK_STATE_NORMAL].blue);
  g_print ("Tooltips bg color is %#4x, %#4x, %#4x\n",
	   style->fg[GTK_STATE_NORMAL].red,
	   style->fg[GTK_STATE_NORMAL].green,
	   style->fg[GTK_STATE_NORMAL].blue);


(For GTK+-1.2, you could create a GtkWindow, use gtk_widget_set_name()
to name it gtk-tooltips, call gtk_widget_ensure_style() and then 
use widget->style.)

Basically, the fact that the tooltips style is attached to a window
named "gtk-tooltips" is public knowledge (otherwise themes couldn't
set this color), the internal details of GtkTooltips are not.


Use gtk_rc_get_style_by_paths() to get the style of tooltip window.

Owen, you are right. After this patch applied, mozilla(gtk2) runs fine with 
tooltips on Solaris . Thank you very much!

Chris, could you please review it?
or can byrner also review this patch? thanks.
Comment on attachment 93252 [details] [diff] [review]
Patch as Owen suggested

r=blizzard

Go ahead and check it in.

(yay!)
Attachment #93252 - Flags: review+
checked in!
Status: ASSIGNED → RESOLVED
Closed: 22 years ago
Resolution: --- → FIXED
Hmm, I think the new code might be leaking the style ... you need
to g_object_unref (style) after you are done with it. (I don't
have the context beyond the patch.)
Crap, I think Owen is right.  I was looking at the wrong function to see if it
was referenced and needed to be dereferenced.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comments from: 
http://developer.gnome.org/doc/API/2.0/gtk/gtk-resource-files.html#gtk-rc-get-style-by-paths
"
Returns : A style created by matching with the supplied paths, or NULL if
nothing matching was specified and the default style should be used. The
returned value is owned by GTK+ as part of an internal cache, so you must call
g_object_ref() on the returned value if you want to keep a reference to it.
"

Do we really need to g_object_unref(style) when it is not reference?
--> Bolian
Assignee: jay.yan → bolian.yin
Status: REOPENED → NEW
Ahh, so it is.
Status: NEW → RESOLVED
Closed: 22 years ago → 22 years ago
Resolution: --- → FIXED
thanks, chris.
and thanks, Jeff, you are working hard even on the last day of staying here.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: