Add debug output to Email Preferences test script

RESOLVED WONTFIX

Status

()

Bugzilla
QA Test Scripts
--
enhancement
RESOLVED WONTFIX
7 years ago
3 years ago

People

(Reporter: wicked, Assigned: wicked)

Tracking

Details

Attachments

(1 attachment, 1 obsolete attachment)

(Assignee)

Description

7 years ago
Created attachment 513705 [details] [diff] [review]
Email Preferences test debug output, V1

Here's a patch that add some debug output for the problematic Email Preference test script. It makes the following changes:
 1) Switches parameters to is_deeply() around to have a more logical output of expected and seen diagnostics if the arrays differ.
 2) Add a separate dump of the arrays to see their full content if an error is detected for a is_deeply() test.
 3) Also adds a dump of the full body text as seen by Selenium.

BTW, this change is actually live for the 3.6 MySQL QA tests to help debug the burning.
Attachment #513705 - Flags: review?(LpSolit)

Comment 1

7 years ago
Comment on attachment 513705 [details] [diff] [review]
Email Preferences test debug output, V1

>+is_deeply(\@email_sentto, \@email_both, "Admin and normal user got bugmail")
>+  or diag(Dumper(\@email_sentto, \@email_both, $sel->get_body_text()));

I'm not a fan to display the whole page. This won't bring any value IMO, besides maybe the CC, assignee and QA fields. I would prefer if you limit the output to these fields only.

Comment 2

7 years ago
Comment on attachment 513705 [details] [diff] [review]
Email Preferences test debug output, V1

Per my previous comment.
Attachment #513705 - Flags: review?(LpSolit) → review-
(Assignee)

Comment 3

7 years ago
Created attachment 515855 [details] [diff] [review]
Email Preferences test debug output, V2

I refactored the bugmail recipient checks to a sub, like you requested on IRC. This also extracts roles from the page in case an error happens.

Screens after setting prefs and error are also stored locally for reference. This change can be left out if you don't want it in repo.
Assignee: testing → wicked
Attachment #513705 - Attachment is obsolete: true
Status: NEW → ASSIGNED
Attachment #515855 - Flags: review?(LpSolit)

Comment 4

7 years ago
Comment on attachment 515855 [details] [diff] [review]
Email Preferences test debug output, V2

As discussed on IRC, this code currently doesn't help us find what's wrong with this QA script on Tinderbox (which is the only place where we found a problem). So I would prefer to limit this patch to Tinderbox for now. You can re-request review later if it really helps us track what's wrong. But for now, I keep it out of my radar.
Attachment #515855 - Flags: review?(LpSolit)

Comment 5

6 years ago
I suspect a race condition could be the reason of the random failures in this script (e.g. Selenium doesn't have time to check a checkbox before changes are submitted). Debugging code won't help as it really depends on the version of Selenium RC and of Firefox you have. So marking as wontfix.
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → WONTFIX
(Assignee)

Updated

3 years ago
Blocks: 1026391
You need to log in before you can comment on or make changes to this bug.