Intermittent toolkit/content/tests/chrome/test_panel_anchoradjust.xul | Test timed out.

Status

()

Toolkit
XUL Widgets
REOPENED
3 months ago
7 days ago

People

(Reporter: Treeherder Bug Filer, Assigned: Neil Deakin (not available until Aug 9), NeedInfo)

Tracking

({intermittent-failure})

unspecified
intermittent-failure
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox52 unaffected, firefox53 wontfix, firefox54 affected, firefox55 affected, firefox-esr52 unaffected)

Details

(Whiteboard: [stockwell fixed])

Attachments

(1 attachment)

(Reporter)

Description

3 months ago
treeherder
Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=78101622&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/XhXNw9nbSm23_3-o0Jt80Q/runs/0/artifacts/public/logs/live_backing.log

Comment 1

3 months ago
10 failures in 812 pushes (0.012 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* autoland: 5
* mozilla-inbound: 4
* mozilla-central: 1

Platform breakdown:
* linux64: 6
* linux32: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-02-20&endday=2017-02-26&tree=all

Comment 2

3 months ago
15 failures in 783 pushes (0.019 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* autoland: 10
* try: 2
* mozilla-inbound: 2
* oak: 1

Platform breakdown:
* linux64: 12
* linux32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-02-27&endday=2017-03-05&tree=all

Comment 3

3 months ago
11 failures in 790 pushes (0.014 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-aurora: 4
* autoland: 4
* mozilla-inbound: 3

Platform breakdown:
* linux32: 6
* linux64: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-03-06&endday=2017-03-12&tree=all

Comment 4

2 months ago
this has picked up in frequency in the last week, 33 failures since last friday :(  Primarily on linux64 debug/asan.

Around March 13th you can see the pickup:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-03-10&endday=2017-03-17&tree=all

doing a few retriggers to help see if there was a change that triggered this to be more frequent:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=linux%20debug%20mochitest-chrome-3%20x64&tochange=950180909fd0e5031a04fcd4cb8f94ff4be72ae6&fromchange=a57d843357e2d93d43e2508de1793f63b355fab9&selectedJob=83437036


I will follow up shortly
Flags: needinfo?(jmaher)
Whiteboard: [stockwell needswork]

Comment 5

2 months ago
shortly is a whole day later!

I ended up doing a few round of retriggers, nothing is standing out as a culprit (orange job == a few different regressions, not always this regression).

the first instance is found here:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2b52802899ae6b4d14481481e0ea37664c48a7d5

I am not sure if that is a root cause since this doesn't happen much, but I have many retriggers on a few revisions prior with no instances.

here is data from the log [0]:
[task 2017-03-17T14:07:27.272930Z] 14:07:27     INFO - TEST-START | toolkit/content/tests/chrome/test_panel.xul
[task 2017-03-17T14:07:27.857787Z] 14:07:27     INFO - GECKO(5499) | MEMORY STAT | vsize 20973989MB | residentFast 1473MB
[task 2017-03-17T14:07:27.875838Z] 14:07:27     INFO - TEST-OK | toolkit/content/tests/chrome/test_panel.xul | took 603ms
[task 2017-03-17T14:07:28.943875Z] 14:07:28     INFO - TEST-START | toolkit/content/tests/chrome/test_panel_anchoradjust.xul
[task 2017-03-17T14:12:34.788378Z] 14:12:34     INFO - TEST-INFO | started process screentopng
[task 2017-03-17T14:12:35.135024Z] 14:12:35     INFO - TEST-INFO | screentopng: exit 0
[task 2017-03-17T14:12:35.135103Z] 14:12:35     INFO - Buffered messages logged at 14:07:29
[task 2017-03-17T14:12:35.135142Z] 14:12:35     INFO - must wait for load
[task 2017-03-17T14:12:35.135177Z] 14:12:35     INFO - must wait for focus
[task 2017-03-17T14:12:35.135231Z] 14:12:35     INFO - TEST-PASS | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | anchor moved x 
[task 2017-03-17T14:12:35.135294Z] 14:12:35     INFO - TEST-PASS | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | anchor moved y 
[task 2017-03-17T14:12:35.135352Z] 14:12:35     INFO - TEST-PASS | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | new anchor x 
[task 2017-03-17T14:12:35.135412Z] 14:12:35     INFO - TEST-PASS | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | new anchor y 
[task 2017-03-17T14:12:35.135476Z] 14:12:35     INFO - TEST-PASS | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | new anchor with offset x 
[task 2017-03-17T14:12:35.135537Z] 14:12:35     INFO - TEST-PASS | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | new anchor with offset y 
[task 2017-03-17T14:12:35.135578Z] 14:12:35     INFO - Buffered messages finished
[task 2017-03-17T14:12:35.136762Z] 14:12:35     INFO - TEST-UNEXPECTED-FAIL | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | Test timed out. 
[task 2017-03-17T14:12:35.137369Z] 14:12:35     INFO - reportError@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:120:7
[task 2017-03-17T14:12:35.137560Z] 14:12:35     INFO - TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:141:7
[task 2017-03-17T14:12:35.138133Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.138770Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.138921Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.139509Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.139656Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.140299Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.140880Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.141064Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.141684Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.142286Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.142448Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.143024Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.143199Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.143787Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.144369Z] 14:12:35     INFO - setTimeout handler*TestRunner._checkForHangs@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:162:5
[task 2017-03-17T14:12:35.144510Z] 14:12:35     INFO - TestRunner.runTests@chrome://mochikit/content/tests/SimpleTest/TestRunner.js:379:5
[task 2017-03-17T14:12:35.145108Z] 14:12:35     INFO - RunSet.runtests@chrome://mochikit/content/tests/SimpleTest/setup.js:190:3
[task 2017-03-17T14:12:35.145269Z] 14:12:35     INFO - RunSet.runall@chrome://mochikit/content/tests/SimpleTest/setup.js:169:5
[task 2017-03-17T14:12:35.145399Z] 14:12:35     INFO - hookupTests@chrome://mochikit/content/tests/SimpleTest/setup.js:262:5
[task 2017-03-17T14:12:35.145497Z] 14:12:35     INFO - parseTestManifest@chrome://mochikit/content/manifestLibrary.js:38:5
[task 2017-03-17T14:12:35.145642Z] 14:12:35     INFO - getTestManifest/req.onload@chrome://mochikit/content/manifestLibrary.js:49:11
[task 2017-03-17T14:12:35.145815Z] 14:12:35     INFO - EventHandlerNonNull*getTestManifest@chrome://mochikit/content/manifestLibrary.js:45:3
[task 2017-03-17T14:12:35.145912Z] 14:12:35     INFO - hookup@chrome://mochikit/content/tests/SimpleTest/setup.js:242:5
[task 2017-03-17T14:12:35.145973Z] 14:12:35     INFO - linkAndHookup@chrome://mochikit/content/harness.xul:59:3
[task 2017-03-17T14:12:35.146064Z] 14:12:35     INFO - parseTestManifest@chrome://mochikit/content/manifestLibrary.js:38:5
[task 2017-03-17T14:12:35.146126Z] 14:12:35     INFO - getTestManifest/req.onload@chrome://mochikit/content/manifestLibrary.js:49:11
[task 2017-03-17T14:12:35.146294Z] 14:12:35     INFO - EventHandlerNonNull*getTestManifest@chrome://mochikit/content/manifestLibrary.js:45:3
[task 2017-03-17T14:12:35.146353Z] 14:12:35     INFO - getTestList@chrome://mochikit/content/chrome-harness.js:260:3
[task 2017-03-17T14:12:35.146443Z] 14:12:35     INFO - loadTests@chrome://mochikit/content/harness.xul:38:3
[task 2017-03-17T14:12:35.146502Z] 14:12:35     INFO - EventListener.handleEvent*@chrome://mochikit/content/harness.xul:62:5
[task 2017-03-17T14:12:35.795599Z] 14:12:35     INFO - GECKO(5499) | MEMORY STAT | vsize 20974174MB | residentFast 1607MB
[task 2017-03-17T14:12:35.797829Z] 14:12:35     INFO - TEST-OK | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | took 306854ms
[task 2017-03-17T14:12:35.932355Z] 14:12:35     INFO - TEST-PASS | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | no follow anchor x 
[task 2017-03-17T14:12:35.932446Z] 14:12:35     INFO - TEST-PASS | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | no follow anchor y 
[task 2017-03-17T14:12:35.932498Z] 14:12:35    ERROR - [SimpleTest.finish()] this test already called finish!
[task 2017-03-17T14:12:35.932577Z] 14:12:35     INFO - TEST-UNEXPECTED-ERROR | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | called finish() multiple times
[task 2017-03-17T14:12:35.932615Z] 14:12:35     INFO - TEST-INFO


and the related screenshot:
https://public-artifacts.taskcluster.net/alLNdL6JTQasYIDWzIFz-Q/0/public/test_info//mozilla-test-fail-screenshot_lYeEHW.png

this is the last test output message we get:
TEST-PASS | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | new anchor with offset y

which aligns with the code here:
https://dxr.mozilla.org/mozilla-central/source/toolkit/content/tests/chrome/window_panel_anchoradjust.xul?q=path%3Awindow_panel_anchoradjust.xul&redirect_type=single#100


typically this test takes <1s:
TEST-OK | toolkit/content/tests/chrome/test_panel_anchoradjust.xul | took 596ms

but we are timing out after 5 minutes :(

I would like to sanity check where we are timing out and if the changeset from bug 1346426 seems possible to trigger this.



[0] https://public-artifacts.taskcluster.net/alLNdL6JTQasYIDWzIFz-Q/0/public/logs/live_backing.log
Blocks: 1346426
Flags: needinfo?(jmaher) → needinfo?(enndeakin)

Comment 6

2 months ago
40 failures in 777 pushes (0.051 failures/push) were associated with this bug in the last 7 days. 

This is the #43 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 17
* autoland: 14
* try: 4
* mozilla-central: 3
* oak: 1
* mozilla-beta: 1

Platform breakdown:
* linux64: 37
* linux32: 2
* linux64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-03-13&endday=2017-03-19&tree=all

Comment 7

2 months ago
This test was added in bug 1109868; the first failure recorded here happened on the same day. Hopefully Neil can make this test more reliable.
Blocks: 1109868
This test is showing two popups, the second one anchored on an element in the first one. The first popup is being hidden and the expectation is that the second popup will get hidden due to the first one not being open anymore.

Testing shows that nsXULPopupManager::UpdatePopupPositions isn't being called on Linux when the test fails. It is fairly easy to workaround in most cases by calling UpdatePopupPositions manually whenever a popup is hidden, and I'll probably add something that does this to fix this test, but it's possible that there is some underlying bug.

I didn't investigate whether nsRefreshDriver::Tick is actually called when it fails, but the part where UpdatePopupPositions is not. Timothy, does this sound like a real bug?
Status: NEW → RESOLVED
Last Resolved: 2 months ago
Flags: needinfo?(enndeakin) → needinfo?(tnikkel)
Resolution: --- → FIXED
(Assignee)

Updated

2 months ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to Neil Deakin from comment #8)
> This test is showing two popups, the second one anchored on an element in
> the first one. The first popup is being hidden and the expectation is that
> the second popup will get hidden due to the first one not being open anymore.
> 
> Testing shows that nsXULPopupManager::UpdatePopupPositions isn't being
> called on Linux when the test fails. It is fairly easy to workaround in most
> cases by calling UpdatePopupPositions manually whenever a popup is hidden,
> and I'll probably add something that does this to fix this test, but it's
> possible that there is some underlying bug.
> 
> I didn't investigate whether nsRefreshDriver::Tick is actually called when
> it fails, but the part where UpdatePopupPositions is not. Timothy, does this
> sound like a real bug?

No. I don't think it'll tick if there is nothing wanting a tick. So your proposed fix sounds good to me.
Flags: needinfo?(tnikkel)
checking in here, this test is still failing at ~50 failures/week, it sounds like we have an idea of how to fix this, would it be realistic to expect that in the next week (by the end of the month)?

Comment 11

2 months ago
58 failures in 898 pushes (0.065 failures/push) were associated with this bug in the last 7 days. 

This is the #17 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 21
* mozilla-inbound: 18
* mozilla-central: 8
* mozilla-beta: 4
* graphics: 3
* try: 2
* mozilla-aurora: 2

Platform breakdown:
* linux64: 49
* linux32: 9

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-03-20&endday=2017-03-26&tree=all
Created attachment 8851549 [details]
update the popup anchors whenever a popup is closed
Assignee: nobody → enndeakin
Attachment #8851549 - Flags: review?(tnikkel)
Attachment #8851549 - Flags: review?(tnikkel) → review+

Comment 13

2 months ago
Pushed by neil@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/16fc5defd30f
check for anchor changes that affect popup positions and visibility whenever a popup is closed, r=tn
https://hg.mozilla.org/mozilla-central/rev/16fc5defd30f
Status: REOPENED → RESOLVED
Last Resolved: 2 months ago2 months ago
status-firefox55: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
status-firefox52: --- → unaffected
status-firefox53: --- → affected
status-firefox54: --- → affected
status-firefox-esr52: --- → unaffected
Please request Aurora/Beta approval on this when you get a chance.
Flags: needinfo?(enndeakin)
Comment on attachment 8851549 [details]
update the popup anchors whenever a popup is closed

Approval Request Comment
[Feature/Bug causing the regression]: minor fix causing intermittent test failure
[User impact if declined]: none
[Is this code covered by automated tests?]: fixes test
[Has the fix been verified in Nightly?]:
[Needs manual test from QE? If yes, steps to reproduce]: no
[List of other uplifts needed for the feature/fix]: none, followup for 1109868 
[Is the change risky?]: no
[Why is the change risky/not risky?]: it just manually calls a function that normally gets called every paint, but doesn't in one edge case on linux
[String changes made/needed]: no
Flags: needinfo?(enndeakin)
Attachment #8851549 - Flags: approval-mozilla-beta?
Attachment #8851549 - Flags: approval-mozilla-aurora?
Comment on attachment 8851549 [details]
update the popup anchors whenever a popup is closed

Fix for regression from 53, let's uplift to beta.
Attachment #8851549 - Flags: approval-mozilla-beta?
Attachment #8851549 - Flags: approval-mozilla-beta+
Attachment #8851549 - Flags: approval-mozilla-aurora?
Attachment #8851549 - Flags: approval-mozilla-aurora+

Comment 18

2 months ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-aurora/rev/92c2bde2cec8
status-firefox54: affected → fixed
Whiteboard: [stockwell needswork] → [stockwell fixed]

Comment 19

2 months ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-beta/rev/f32cb099303d
status-firefox53: affected → fixed

Comment 20

2 months ago
56 failures in 845 pushes (0.066 failures/push) were associated with this bug in the last 7 days. 

This is the #28 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 31
* mozilla-inbound: 11
* mozilla-beta: 5
* mozilla-aurora: 4
* try: 2
* mozilla-central: 2
* graphics: 1

Platform breakdown:
* linux64: 44
* linux32: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-03-27&endday=2017-04-02&tree=all
This is still happening :(
Status: RESOLVED → REOPENED
status-firefox53: fixed → affected
status-firefox54: fixed → affected
status-firefox55: fixed → affected
Flags: needinfo?(enndeakin)
Resolution: FIXED → ---
Target Milestone: mozilla55 → ---

Comment 22

2 months ago
16 failures in 867 pushes (0.018 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 5
* mozilla-inbound: 4
* mozilla-aurora: 4
* oak: 1
* mozilla-central: 1
* mozilla-beta: 1

Platform breakdown:
* linux64: 15
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-04-03&endday=2017-04-09&tree=all
status-firefox53: affected → wontfix

Comment 23

a month ago
16 failures in 894 pushes (0.018 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 7
* autoland: 5
* mozilla-central: 3
* try: 1

Platform breakdown:
* linux64: 14
* linux64-nightly: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-04-10&endday=2017-04-16&tree=all

Comment 24

a month ago
7 failures in 817 pushes (0.009 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 2
* try: 1
* mozilla-release: 1
* mozilla-central: 1
* graphics: 1
* autoland: 1

Platform breakdown:
* linux64: 6
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-04-17&endday=2017-04-23&tree=all

Comment 25

28 days ago
9 failures in 883 pushes (0.01 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-beta: 3
* mozilla-inbound: 2
* graphics: 2
* mozilla-central: 1
* autoland: 1

Platform breakdown:
* linux64: 8
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-04-24&endday=2017-04-30&tree=all

Comment 26

21 days ago
16 failures in 770 pushes (0.021 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 10
* mozilla-inbound: 5
* graphics: 1

Platform breakdown:
* linux64: 15
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-05-01&endday=2017-05-07&tree=all

Comment 27

14 days ago
5 failures in 879 pushes (0.006 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 2
* mozilla-central: 1
* mozilla-beta: 1
* autoland: 1

Platform breakdown:
* linux64: 3
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-05-08&endday=2017-05-14&tree=all
14 failures in 777 pushes (0.018 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 6
* autoland: 4
* mozilla-central: 3
* try: 1

Platform breakdown:
* linux32: 10
* linux64: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340413&startday=2017-05-15&endday=2017-05-21&tree=all
You need to log in before you can comment on or make changes to this bug.