Closed Bug 1398685 Opened 7 years ago Closed 6 years ago

Intermittent test_loader_global_sharing.py TestLoaderGlobalSharing.test_global_sharing_settings | IOError: Process killed because the connection to Marionette server is lost. Check gecko.log for errors (Reason: Timed out waiting for connection

Categories

(Core :: XPConnect, defect, P5)

defect

Tracking

()

RESOLVED INCOMPLETE
Tracking Status
firefox57 --- affected
firefox58 --- affected
firefox59 --- affected

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure)

Filed by: archaeopteryx [at] coole-files.de https://treeherder.mozilla.org/logviewer.html#?job_id=129971876&repo=autoland https://queue.taskcluster.net/v1/task/Pd5LynntQMCS-EiABpBLyw/runs/0/artifacts/public/logs/live_backing.log [task 2017-09-11T02:50:50.653242Z] 02:50:50 INFO - 1505098250646 Marionette TRACE 2 -> [0,22,"executeScript",{"scriptTimeout":null,"newSandbox":true,"args":["jsloader.shareGlobal",false,false],"filename":"marionette.py","script":"\n Components.utils.import(\"resource://gre/modules/Preferences.jsm\");\n\n let pref = arguments[0];\n let value = arguments[1];\n let defaultBranch = arguments[2];\n\n prefs = new Preferences({defaultBranch: defaultBranch});\n prefs.set(pref, value);\n ","sandbox":"default","line":928}] [task 2017-09-11T02:50:50.654793Z] 02:50:50 INFO - 1505098250651 Marionette TRACE 2 <- [1,22,null,{"value":null}] [task 2017-09-11T02:50:50.655884Z] 02:50:50 INFO - 1505098250652 Marionette TRACE 2 -> [0,23,"setContext",{"value":"chrome"}] [task 2017-09-11T02:50:50.656875Z] 02:50:50 INFO - 1505098250653 Marionette TRACE 2 <- [1,23,null,{}] [task 2017-09-11T02:50:50.658562Z] 02:50:50 INFO - 1505098250656 Marionette TRACE 2 -> [0,24,"getContext",{}] [task 2017-09-11T02:50:50.659605Z] 02:50:50 INFO - 1505098250657 Marionette TRACE 2 <- [1,24,null,{"value":"chrome"}] [task 2017-09-11T02:50:50.663703Z] 02:50:50 INFO - 1505098250659 Marionette TRACE 2 -> [0,25,"setContext",{"value":"chrome"}] [task 2017-09-11T02:50:50.664693Z] 02:50:50 INFO - 1505098250660 Marionette TRACE 2 <- [1,25,null,{}] [task 2017-09-11T02:50:50.665795Z] 02:50:50 INFO - 1505098250662 Marionette TRACE 2 -> [0,26,"executeScript",{"scriptTimeout":null,"newSandbox":false,"args":["MOZ_LOADER_SHARE_GLOBAL","1"],"filename":"test_loader_global_sharing.py","script":"env.set(arguments[0], arguments[1]);","sandbox":"loader-global-sharing","line":18}] [task 2017-09-11T02:50:50.667798Z] 02:50:50 INFO - 1505098250665 Marionette TRACE 2 <- [1,26,null,{"value":null}] [task 2017-09-11T02:50:50.668975Z] 02:50:50 INFO - 1505098250667 Marionette TRACE 2 -> [0,27,"setContext",{"value":"chrome"}] [task 2017-09-11T02:50:50.669964Z] 02:50:50 INFO - 1505098250667 Marionette TRACE 2 <- [1,27,null,{}] [task 2017-09-11T02:50:50.671625Z] 02:50:50 INFO - 1505098250669 Marionette TRACE 2 -> [0,28,"getContext",{}] [task 2017-09-11T02:50:50.672509Z] 02:50:50 INFO - 1505098250670 Marionette TRACE 2 <- [1,28,null,{"value":"chrome"}] [task 2017-09-11T02:50:50.676661Z] 02:50:50 INFO - 1505098250672 Marionette TRACE 2 -> [0,29,"getContext",{}] [task 2017-09-11T02:50:50.677935Z] 02:50:50 INFO - 1505098250673 Marionette TRACE 2 <- [1,29,null,{"value":"chrome"}] [task 2017-09-11T02:50:50.679376Z] 02:50:50 INFO - 1505098250674 Marionette TRACE 2 -> [0,30,"setContext",{"value":"chrome"}] [task 2017-09-11T02:50:50.680892Z] 02:50:50 INFO - 1505098250675 Marionette TRACE 2 <- [1,30,null,{}] [task 2017-09-11T02:50:50.682404Z] 02:50:50 INFO - 1505098250676 Marionette TRACE 2 -> [0,31,"executeScript",{"scriptTimeout":null,"newSandbox":true,"args":[],"filename":"marionette.py","script":"\n Components.utils.import(\"resource://gre/modules/Services.jsm\");\n let cancelQuit = Components.classes[\"@mozilla.org/supports-PRBool;1\"]\n .createInstance(Components.interfaces.nsISupportsPRBool);\n Services.obs.notifyObservers(cancelQuit, \"quit-application-requested\", null);\n return cancelQuit.data;\n ","sandbox":"default","line":1061}] [task 2017-09-11T02:50:50.699051Z] 02:50:50 INFO - 1505098250687 Marionette TRACE 2 <- [1,31,null,{"value":false}] [task 2017-09-11T02:50:50.699995Z] 02:50:50 INFO - 1505098250688 Marionette TRACE 2 -> [0,32,"setContext",{"value":"chrome"}] [task 2017-09-11T02:50:50.701003Z] 02:50:50 INFO - 1505098250689 Marionette TRACE 2 <- [1,32,null,{}] [task 2017-09-11T02:50:50.702002Z] 02:50:50 INFO - 1505098250689 Marionette TRACE 2 -> [0,33,"quitApplication",{"flags":["eRestart","eAttemptQuit"]}] [task 2017-09-11T02:50:50.702871Z] 02:50:50 INFO - 1505098250690 Marionette INFO New connections will no longer be accepted [task 2017-09-11T02:50:50.856463Z] 02:50:50 INFO - 1505098250842 Marionette TRACE 2 <- [1,33,null,{"cause":"restart"}] [task 2017-09-11T02:50:50.864213Z] 02:50:50 INFO - 1505098250859 Marionette WARN New connections are currently not accepted [task 2017-09-11T02:50:50.881355Z] 02:50:50 INFO - 1505098250869 Marionette DEBUG Closed connection 2 [task 2017-09-11T02:50:51.167730Z] 02:50:51 INFO - *** UTM:SVC TimerManager:registerTimer called after profile-before-change notification. Ignoring timer registration for id: telemetry_modules_ping [task 2017-09-11T02:50:51.332752Z] 02:50:51 INFO - 1505098251330 Marionette DEBUG Received observer notification "xpcom-shutdown" [task 2017-09-11T02:56:52.021098Z] 02:56:52 INFO - TEST-UNEXPECTED-ERROR | test_loader_global_sharing.py TestLoaderGlobalSharing.test_global_sharing_settings | IOError: Process killed because the connection to Marionette server is lost. Check gecko.log for errors (Reason: Timed out waiting for connection on localhost:2828!)
This looks like a hang during final shutdown. Means anything which comes after "xpcom-shutdown". As such Marionette is not able to reconnect to the browser given that it hasn't been restarted in 6 mins!
Component: Marionette → XPConnect
Product: Testing → Core
Version: Version 3 → unspecified
See Also: → 1398601
I've noticed in some other failures that BackgroundPageThumbs.jsm tends to deadlock the main thread in short sessions, and it's still running during shutdown. That might be the cause of this too.
Depends on: 1399646
In a debug build I can see the following in the log: 21:24:57 INFO - Assertion failure: 0 == rv, at /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:870 21:24:57 INFO - Redirecting call to abort() to mozalloc_abort 21:24:57 INFO - Hit MOZ_CRASH() at /builds/worker/workspace/build/src/memory/mozalloc/mozalloc_abort.cpp:33 21:24:57 INFO - Assertion failure: 0 == rv, at /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:292 21:24:57 INFO - Redirecting call to abort() to mozalloc_abort 21:24:57 INFO - Hit MOZ_CRASH() at /builds/worker/workspace/build/src/memory/mozalloc/mozalloc_abort.cpp:33
The failure here happens during an in_app restart of Firefox. So it might be fixed with my patch on bug 1410366 landed.
Depends on: 1410366
Bug 1399646 was fixed about a month ago, so it clearly didn't fix the problem. I will wait a couple of days and then check if my patch on bug 1410366 fixed it.
No longer depends on: 1399646
All of the current failures are happening for ASAN builds, and most of them show the following output in the log: [task 2017-11-28T02:34:37.658Z] 02:34:37 INFO - AddressSanitizer:DEADLYSIGNAL [task 2017-11-28T02:34:37.659Z] 02:34:37 INFO - ================================================================= [task 2017-11-28T02:34:37.659Z] 02:34:37 ERROR - ==5175==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fc48befc135 bp 0x7fc457bfedf0 sp 0x7fc457bfede0 T70) [task 2017-11-28T02:34:37.660Z] 02:34:37 INFO - ==5175==The signal is caused by a WRITE memory access. [task 2017-11-28T02:34:37.661Z] 02:34:37 INFO - ==5175==Hint: address points to the zero page. [task 2017-11-28T02:34:38.468Z] 02:34:38 INFO - #0 0x7fc48befc134 in mozilla::(anonymous namespace)::RunWatchdog(void*) /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:168:5 [task 2017-11-28T02:34:38.469Z] 02:34:38 INFO - #1 0x7fc49cad707e in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:216:5 [task 2017-11-28T02:34:38.470Z] 02:34:38 INFO - #2 0x7fc4a042b6b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9) [task 2017-11-28T02:34:38.527Z] 02:34:38 INFO - #3 0x7fc49f4b43dc in clone /build/glibc-bfm8X4/glibc-2.23/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:109 [task 2017-11-28T02:34:38.528Z] 02:34:38 INFO - AddressSanitizer can not provide additional info. [task 2017-11-28T02:34:38.529Z] 02:34:38 INFO - SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:168:5 in mozilla::(anonymous namespace)::RunWatchdog(void*) [task 2017-11-28T02:34:38.530Z] 02:34:38 INFO - Thread T70 (Shutdow~minator) created by T0 here: [task 2017-11-28T02:34:38.546Z] 02:34:38 INFO - #0 0x4a816d in __interceptor_pthread_create /builds/worker/workspace/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:204:3 [task 2017-11-28T02:34:38.547Z] 02:34:38 INFO - #1 0x7fc49cad3dcf in _PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:457:14 [task 2017-11-28T02:34:38.548Z] 02:34:38 INFO - #2 0x7fc49cad39be in PR_CreateThread /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:548:12 [task 2017-11-28T02:34:38.548Z] 02:34:38 INFO - #3 0x7fc48befc379 in CreateSystemThread /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:74:22 [task 2017-11-28T02:34:38.549Z] 02:34:38 INFO - #4 0x7fc48befc379 in StartWatchdog /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:409 [task 2017-11-28T02:34:38.550Z] 02:34:38 INFO - #5 0x7fc48befc379 in Start /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:369 [task 2017-11-28T02:34:38.551Z] 02:34:38 INFO - #6 0x7fc48befc379 in mozilla::nsTerminator::Observe(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:464 [task 2017-11-28T02:34:38.551Z] 02:34:38 INFO - #7 0x7fc4804d403c in nsObserverList::NotifyObservers(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/xpcom/ds/nsObserverList.cpp:112:19 [task 2017-11-28T02:34:38.552Z] 02:34:38 INFO - #8 0x7fc4804d7fcc in nsObserverService::NotifyObservers(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/xpcom/ds/nsObserverService.cpp:296:19 [task 2017-11-28T02:34:38.556Z] 02:34:38 INFO - #9 0x7fc48bd37e67 in nsAppStartup::Quit(unsigned int) /builds/worker/workspace/build/src/toolkit/components/startup/nsAppStartup.cpp:466:19 [task 2017-11-28T02:34:38.557Z] 02:34:38 INFO - #10 0x7fc48bd39ff7 in ExitLastWindowClosingSurvivalArea /builds/worker/workspace/build/src/toolkit/components/startup/nsAppStartup.cpp:538:5 [task 2017-11-28T02:34:38.559Z] 02:34:38 INFO - #11 0x7fc48bd39ff7 in nsAppStartup::Observe(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/toolkit/components/startup/nsAppStartup.cpp:712 [task 2017-11-28T02:34:38.560Z] 02:34:38 INFO - #12 0x7fc48bd3a11c in non-virtual thunk to nsAppStartup::Observe(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/toolkit/components/startup/nsAppStartup.cpp [task 2017-11-28T02:34:38.561Z] 02:34:38 INFO - #13 0x7fc4804d403c in nsObserverList::NotifyObservers(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/xpcom/ds/nsObserverList.cpp:112:19 [task 2017-11-28T02:34:38.562Z] 02:34:38 INFO - #14 0x7fc4804d7fcc in nsObserverService::NotifyObservers(nsISupports*, char const*, char16_t const*) /builds/worker/workspace/build/src/xpcom/ds/nsObserverService.cpp:296:19 [task 2017-11-28T02:34:38.579Z] 02:34:38 INFO - #15 0x7fc48b55ef21 in nsXULWindow::Destroy() /builds/worker/workspace/build/src/xpfe/appshell/nsXULWindow.cpp:526:15 [task 2017-11-28T02:34:38.579Z] 02:34:38 INFO - #16 0x7fc48b535d5b in nsWebShellWindow::Destroy() /builds/worker/workspace/build/src/xpfe/appshell/nsWebShellWindow.cpp:783:23 [task 2017-11-28T02:34:38.580Z] 02:34:38 INFO - #17 0x7fc48342a80a in nsGlobalWindowOuter::ReallyCloseWindow() /builds/worker/workspace/build/src/dom/base/nsGlobalWindowOuter.cpp:6138:27 [task 2017-11-28T02:34:38.581Z] 02:34:38 INFO - #18 0x7fc48343722c in nsCloseEvent::Run() /builds/worker/workspace/build/src/dom/base/nsGlobalWindowOuter.cpp:5911:16 [task 2017-11-28T02:34:38.597Z] 02:34:38 INFO - #19 0x7fc4805da33e in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp:1033:14 [task 2017-11-28T02:34:38.598Z] 02:34:38 INFO - #20 0x7fc4805f60c0 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/workspace/build/src/xpcom/threads/nsThreadUtils.cpp:508:10 [task 2017-11-28T02:34:38.615Z] 02:34:38 INFO - #21 0x7fc4814670fa in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /builds/worker/workspace/build/src/ipc/glue/MessagePump.cpp:97:21 [task 2017-11-28T02:34:38.616Z] 02:34:38 INFO - #22 0x7fc4813be399 in RunInternal /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:326:10 [task 2017-11-28T02:34:38.617Z] 02:34:38 INFO - #23 0x7fc4813be399 in RunHandler /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:319 [task 2017-11-28T02:34:38.619Z] 02:34:38 INFO - #24 0x7fc4813be399 in MessageLoop::Run() /builds/worker/workspace/build/src/ipc/chromium/src/base/message_loop.cc:299 [task 2017-11-28T02:34:38.640Z] 02:34:38 INFO - #25 0x7fc48781b58a in nsBaseAppShell::Run() /builds/worker/workspace/build/src/widget/nsBaseAppShell.cpp:157:27 [task 2017-11-28T02:34:38.641Z] 02:34:38 INFO - #26 0x7fc48bd370eb in nsAppStartup::Run() /builds/worker/workspace/build/src/toolkit/components/startup/nsAppStartup.cpp:288:30 [task 2017-11-28T02:34:38.642Z] 02:34:38 INFO - #27 0x7fc48bf4f958 in XREMain::XRE_mainRun() /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4649:22 [task 2017-11-28T02:34:38.642Z] 02:34:38 INFO - #28 0x7fc48bf5278e in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4811:8 [task 2017-11-28T02:34:38.643Z] 02:34:38 INFO - #29 0x7fc48bf53c04 in XRE_main(int, char**, mozilla::BootstrapConfig const&) /builds/worker/workspace/build/src/toolkit/xre/nsAppRunner.cpp:4903:21 [task 2017-11-28T02:34:38.644Z] 02:34:38 INFO - #30 0x4ee80b in do_main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:231:22 [task 2017-11-28T02:34:38.644Z] 02:34:38 INFO - #31 0x4ee80b in main /builds/worker/workspace/build/src/browser/app/nsBrowserApp.cpp:304 [task 2017-11-28T02:34:38.645Z] 02:34:38 INFO - #32 0x7fc49f3cd82f in __libc_start_main /build/glibc-bfm8X4/glibc-2.23/csu/../csu/libc-start.c:291 [task 2017-11-28T02:34:38.645Z] 02:34:38 INFO - ==5175==ABORTING Kris, could you have a look at this? For now I'm going to mark this bug as security related.
Group: core-security
Flags: needinfo?(kmaglione+bmo)
My best guess so far is that this is just because we're shutting down *very* soon after startup, and running into some odd corner case. In any case, I don't think this bug needs to be hidden. This is just a watchdog timeout because shutdown is taking too long. It's definitely not exploitable.
Flags: needinfo?(kmaglione+bmo)
Group: core-security
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INCOMPLETE
Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=183540918&repo=mozilla-inbound&lineNumber=50546 11:43:34 INFO - 1529235814532 Marionette TRACE 3 -> [0,18,"WebDriver:ExecuteScript",{"scriptTimeout":null,"newSandbox":false,"args":[],"filename":"Z:\\task_1529234494\\build\\te ... lForObject(Services) ===\n Cu.getGlobalForObject(XPCOMUtils))","sandbox":"loader-global-sharing","line":18}] 11:43:34 INFO - 1529235814535 Marionette TRACE 3 <- [1,18,null,{"value":true}] 11:43:34 INFO - 1529235814537 Marionette TRACE 3 -> [0,19,"Marionette:SetContext",{"value":"chrome"}] 11:43:34 INFO - 1529235814537 Marionette TRACE 3 <- [1,19,null,{"value":null}] 11:43:34 INFO - 1529235814539 Marionette TRACE 3 -> [0,20,"Marionette:GetContext",{}] 11:43:34 INFO - 1529235814540 Marionette TRACE 3 <- [1,20,null,{"value":"chrome"}] 11:43:34 INFO - 1529235814541 Marionette TRACE 3 -> [0,21,"WebDriver:DeleteSession",{}] 11:43:34 INFO - 1529235814544 Marionette TRACE 3 <- [1,21,null,{"value":null}] 11:43:34 INFO - Application command: Z:\task_1529234494\build\application\firefox\firefox.exe -no-remote -marionette -profile c:\users\task_1529234494\appdata\local\temp\tmphj6ibz.mozrunner 11:43:36 INFO - 1529235816847 Marionette DEBUG Received observer notification profile-after-change 11:43:37 INFO - 1529235817061 Marionette DEBUG Received observer notification command-line-startup 11:43:37 INFO - 1529235817062 Marionette DEBUG Received observer notification nsPref:changed 11:43:37 INFO - 1529235817062 Marionette DEBUG Init aborted (running=false, enabled=true, finalUIStartup=false) 11:47:36 ERROR - TEST-UNEXPECTED-ERROR | js\xpconnect\tests\marionette\test_loader_global_sharing.py TestLoaderGlobalSharing.test_global_sharing_settings | IOError: Process killed because the connection to Marionette server is lost. Check gecko.log for errors (Reason: Timed out waiting for connection on localhost:2828!) 11:47:36 INFO - Traceback (most recent call last): 11:47:36 INFO - File "Z:\task_1529234494\build\venv\lib\site-packages\marionette_harness\marionette_test\testcases.py", line 190, in run 11:47:36 INFO - self.tearDown() 11:47:36 INFO - File "Z:\task_1529234494\build\tests\marionette\tests\js\xpconnect\tests\marionette\test_loader_global_sharing.py", line 77, in tearDown 11:47:36 INFO - self.marionette.restart(clean=True) 11:47:36 INFO - File "Z:\task_1529234494\build\venv\lib\site-packages\marionette_driver\decorators.py", line 31, in _ 11:47:36 INFO - m._handle_socket_failure() 11:47:36 INFO - File "Z:\task_1529234494\build\venv\lib\site-packages\marionette_driver\marionette.py", line 824, in _handle_socket_failure 11:47:36 INFO - reraise(IOError, message.format(returncode=returncode, reason=val), tb) 11:47:36 INFO - File "Z:\task_1529234494\build\venv\lib\site-packages\marionette_driver\decorators.py", line 26, in _ 11:47:36 INFO - return func(*args, **kwargs) 11:47:36 INFO - File "Z:\task_1529234494\build\venv\lib\site-packages\marionette_driver\marionette.py", line 1162, in restart 11:47:36 INFO - self.raise_for_port(timeout=self.DEFAULT_STARTUP_TIMEOUT) 11:47:36 INFO - File "Z:\task_1529234494\build\venv\lib\site-packages\marionette_driver\marionette.py", line 710, in raise_for_port 11:47:36 INFO - self.host, self.port)) 11:47:36 INFO - TEST-INFO took 287312ms 11:47:36 INFO - SUMMARY 11:47:36 INFO - ------- 11:47:36 INFO - passed: 850 11:47:36 INFO - failed: 1 11:47:36 INFO - todo: 37 (skipped: 32) 11:47:36 INFO - FAILED TESTS 11:47:36 INFO - ------- 11:47:36 INFO - test_loader_global_sharing.py test_loader_global_sharing.TestLoaderGlobalSharing.test_global_sharing_settings 11:47:36 INFO - SUITE-END | took 1103s 11:47:36 ERROR - Return code: 10 11:47:36 INFO - TinderboxPrint: marionette<br/>850/<em class="testfail">1</em>/37 11:47:36 INFO - gecko.log not found 11:47:36 INFO - TinderboxPrint: marionette<br/>850/<em class="testfail">1</em>/37 11:47:36 INFO - Marionette exited with return code 10: WARNING 11:47:36 WARNING - # TBPL WARNING # 11:47:36 WARNING - setting return code to 1
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 7 years ago6 years ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.