1940773 - Perma [tier 2] toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js | uncaught exception - TypeError: can't access property "todo", SpecialPowers.SimpleTest is undefined at a11yWarn@chrome://mochikit/content/tests/SimpleTest/

Status: NEW

(Toolkit :: Add-ons Manager, defect)

Description

[Treeherder Bug Filer]

Filed by: chorotan [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=489404646&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VWeE0r9kT_S-Dx8xDpavQQ/runs/0/artifacts/public/logs/live_backing.log

---

task 2025-01-09T15:29:24.778Z] 15:29:24     INFO - TEST-PASS | toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js | Initial checkbox state - 
[task 2025-01-09T15:29:24.778Z] 15:29:24     INFO - Buffered messages finished
[task 2025-01-09T15:29:24.781Z] 15:29:24     INFO - TEST-UNEXPECTED-FAIL | toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js | uncaught exception - TypeError: can't access property "todo", SpecialPowers.SimpleTest is undefined at a11yWarn@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:768:5
[task 2025-01-09T15:29:24.781Z] 15:29:24     INFO - assertLabelled@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:981:17
[task 2025-01-09T15:29:24.781Z] 15:29:24     INFO - assertCanBeClicked@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:1176:21
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - runIfA11YChecks/this.AccessibilityUtils@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:1140:40
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - handleEvent@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:1257:12
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - toggleUserScriptCheckbox@chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js:116:12
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - verifyUserScriptPromptDeniedByDefault@chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js:165:31
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - async*test_userScripts_not_allowed_by_default@chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js:203:9
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - async*handleTask@chrome://mochikit/content/browser-test.js:1147:26
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - _runTaskBasedTest@chrome://mochikit/content/browser-test.js:1219:18
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - async*Tester_execTest@chrome://mochikit/content/browser-test.js:1360:14
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - nextTest/<@chrome://mochikit/content/browser-test.js:1136:14
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - SimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - 
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - Stack trace:
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:simpletestOnerror:2068
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js:toggleUserScriptCheckbox:116
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js:verifyUserScriptPromptDeniedByDefault:165
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js:test_userScripts_not_allowed_by_default:203
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - chrome://mochikit/content/browser-test.js:handleTask:1147
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - chrome://mochikit/content/browser-test.js:_runTaskBasedTest:1219
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1360
[task 2025-01-09T15:29:24.782Z] 15:29:24     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:1136
[task 2025-01-09T15:29:24.783Z] 15:29:24     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/<:1058
[task 2025-01-09T15:29:24.783Z] 15:29:24     INFO - GECKO(12767) | JavaScript error: chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js, line 768: TypeError: can't access property "todo", SpecialPowers.SimpleTest is undefined
[task 2025-01-09T15:29:24.784Z] 15:29:24     INFO - TEST-PASS | toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js | Checked state should change - 
[task 2025-01-09T15:29:24.784Z] 15:29:24     INFO - TEST-PASS | toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js | Should be able to Deny the prompt - 
[task 2025-01-09T15:29:24.786Z] 15:29:24     INFO - Console message: [JavaScript Error: "TypeError: can't access property "todo", SpecialPowers.SimpleTest is undefined" {file: "chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js" line: 768}]
[task 2025-01-09T15:29:24.786Z] 15:29:24     INFO - a11yWarn@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:768:5
[task 2025-01-09T15:29:24.786Z] 15:29:24     INFO - assertLabelled@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:981:17
[task 2025-01-09T15:29:24.787Z] 15:29:24     INFO - assertCanBeClicked@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:1176:21
[task 2025-01-09T15:29:24.787Z] 15:29:24     INFO - runIfA11YChecks/this.AccessibilityUtils@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:1140:40
[task 2025-01-09T15:29:24.787Z] 15:29:24     INFO - handleEvent@chrome://mochikit/content/tests/SimpleTest/AccessibilityUtils.js:1257:12
[task 2025-01-09T15:29:24.787Z] 15:29:24     INFO - toggleUserScriptCheckbox@chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js:116:12
[task 2025-01-09T15:29:24.787Z] 15:29:24     INFO - verifyUserScriptPromptDeniedByDefault@chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js:165:31
[task 2025-01-09T15:29:24.787Z] 15:29:24     INFO - async*test_userScripts_not_allowed_by_default@chrome://mochitests/content/browser/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js:203:9
[task 2025-01-09T15:29:24.788Z] 15:29:24     INFO - async*handleTask@chrome://mochikit/content/browser-test.js:1147:26
[task 2025-01-09T15:29:24.788Z] 15:29:24     INFO - _runTaskBasedTest@chrome://mochikit/content/browser-test.js:1219:18
[task 2025-01-09T15:29:24.788Z] 15:29:24     INFO - async*Tester_execTest@chrome://mochikit/content/browser-test.js:1360:14
[task 2025-01-09T15:29:24.788Z] 15:29:24     INFO - nextTest/<@chrome://mochikit/content/browser-test.js:1136:14
[task 2025-01-09T15:29:24.788Z] 15:29:24     INFO - SimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13
[task 2025-01-09T15:29:24.788Z] 15:29:24     INFO - 

Comment 1

[Intermittent Failures Robot]

Summary

15 failures in 700 pushes (0.021 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 13
• try: 1
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-qr: 9
  • opt: 9
• linux1804-64-shippable-qr: 6
  • opt: 6

Table

	a11y-checks-swr
linux1804-64-qr/opt	9
linux1804-64-shippable-qr/opt	6

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1940773&startday=2025-01-10&endday=2025-01-10&tree=all

Comment 2

[Intermittent Failures Robot]

Summary

36 failures in 3682 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This failure happened more than 30 times this week! Resolving this bug is a high priority.

Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled.

Repository breakdown:

• autoland: 31
• try: 1
• mozilla-central: 4

Platform and build breakdown:

• linux1804-64-qr: 22
  • opt: 22
• linux1804-64-shippable-qr: 14
  • opt: 14

Table

	a11y-checks-swr
linux1804-64-qr/opt	22
linux1804-64-shippable-qr/opt	14

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1940773&startday=2025-01-06&endday=2025-01-12&tree=all

Comment 3

[amarc]

Hi Rob! Could this permafailure be caused by your changes made on Bug 1931548 ? If so, could you please take a look? Thanks.

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1931548

Comment 5

[Rob Wu [:robwu]]

While bug 1931548 added a new test, I am not sure if this is really an issue with my patches.

I simulate a checkbox change with checkbox.click() at https://searchfox.org/mozilla-central/rev/9f9fa5d2cb72b12ac7e168b7f6ee9820f63291e9/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js#115-116.
The checkbox is visible, with a label - nothing special: https://searchfox.org/mozilla-central/rev/9f9fa5d2cb72b12ac7e168b7f6ee9820f63291e9/browser/base/content/browser-addons.js#345-351

When a11y checks are enabled, that apparently triggers an error at https://searchfox.org/mozilla-central/rev/9f9fa5d2cb72b12ac7e168b7f6ee9820f63291e9/testing/mochitest/tests/SimpleTest/AccessibilityUtils.js#755. This error is independent of my test.
The error is in an attempt to call SimpleTest.todo, which should have been a non-fatal error. The intended result is to log "Form elements should have a visible text label" at https://searchfox.org/mozilla-central/rev/9f9fa5d2cb72b12ac7e168b7f6ee9820f63291e9/testing/mochitest/tests/SimpleTest/AccessibilityUtils.js#981

Oddly, when I run the test on macOS, I don't get any failures (nor warnings):

./mach test toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js --headless --log-mach-verbose --enable-a11y-checks

Comment 6

[Rob Wu [:robwu]]

Anna, see comment 5 and the summary below. Do you know what is going on here?

• <checkbox> with label created at https://searchfox.org/mozilla-central/rev/9f9fa5d2cb72b12ac7e168b7f6ee9820f63291e9/browser/base/content/browser-addons.js#345-351
• <checkbox> custom element defined at https://searchfox.org/mozilla-central/source/toolkit/content/widgets/checkbox.js
• Checkbox toggled by checkbox.click() at https://searchfox.org/mozilla-central/rev/9f9fa5d2cb72b12ac7e168b7f6ee9820f63291e9/toolkit/mozapps/extensions/test/browser/browser_permission_prompt_userScripts.js#115-116

For some reason, AccessibilityUtils.js thinks that the checkbox does not have a form label, on Linux only (not on macOS), at https://searchfox.org/mozilla-central/rev/9f9fa5d2cb72b12ac7e168b7f6ee9820f63291e9/testing/mochitest/tests/SimpleTest/AccessibilityUtils.js#981.
That would ordinarily have been a non-fatal warning, but somehow SpecialPowers.SimpleTest is undefined at that point, causing a failure upon calling SpecialPowers.SimpleTest.todo at https://searchfox.org/mozilla-central/rev/9f9fa5d2cb72b12ac7e168b7f6ee9820f63291e9/testing/mochitest/tests/SimpleTest/AccessibilityUtils.js#755

Comment 7

[Intermittent Failures Robot]

Summary

16 failures in 693 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 14
• mozilla-central: 2

Platform and build breakdown:

• linux1804-64-qr: 10
  • opt: 10
• linux1804-64-shippable-qr: 6
  • opt: 6

Table

	a11y-checks-swr
linux1804-64-qr/opt	10
linux1804-64-shippable-qr/opt	6

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1940773&startday=2025-01-13&endday=2025-01-13&tree=all

Comment 8

[Intermittent Failures Robot]

Summary

16 failures in 926 pushes (0.017 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 15
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-qr: 16
  • opt: 16

Table

	a11y-checks-swr
linux1804-64-qr/-/opt	16

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1940773&startday=2025-01-16&endday=2025-01-16&tree=all

Comment 9

[Intermittent Failures Robot]

Summary

69 failures in 4508 pushes (0.015 failures/push) were associated with this bug in the last 7 days.

This is the #15 most frequent failure this week.

This failure happened more than 30 times this week! Resolving this bug is a high priority.

Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled.

Repository breakdown:

• autoland: 56
• try: 5
• mozilla-central: 8

Platform and build breakdown:

• linux1804-64-qr: 68
  • opt: 68
• linux2204-64-wayland: 1
  • opt: 1

Table

	a11y-checks	a11y-checks-swr	no_variant
linux1804-64-qr/-/opt	2	66
linux2204-64-wayland/-/opt			1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1940773&startday=2025-01-13&endday=2025-01-19&tree=all

Comment 10

[Rob Wu [:robwu]]

This is not unique to browser_permission_prompt_userScripts.js, here is a try push with the same error for browser_installtrigger_install.js.

https://treeherder.mozilla.org/jobs?repo=try&revision=7c45e3468d6060973d8078dbab6da50e4b980b15&selectedTaskRun=IFh7vpc2StarkBMl4Yv10w.0

Comment 11

[Intermittent Failures Robot]

Summary

15 failures in 756 pushes (0.02 failures/push) were associated with this bug yesterday.

This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed.

Repository breakdown:

• autoland: 13
• mozilla-central: 2

Platform and build breakdown:

• linux1804-64-qr: 15
  • opt: 15

Table

	a11y-checks-swr
linux1804-64-qr--/opt	15

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1940773&startday=2025-01-23&endday=2025-01-23&tree=all

Comment 12

[Rob Wu [:robwu]]

Attachment: Bug 1940773 - Skip a11y_checks in browser_permission_prompt_userScripts.js

This test currently permafails, see
https://bugzilla.mozilla.org/show_bug.cgi?id=1940773#c5 and
https://bugzilla.mozilla.org/show_bug.cgi?id=1940773#c6

Disabling test on a11y_checks until the underlying issue is addressed.

Comment 13

[Rob Wu [:robwu]]

I'm going to skip the test when running with a11y_checks, since the failures are not helpful. There is a pending needinfo in comment 6 that blocks the "proper" resolution of this bug.

Comment 14

[Intermittent Failures Robot]

Summary

68 failures in 4035 pushes (0.017 failures/push) were associated with this bug in the last 7 days.

This is the #17 most frequent failure this week.

This failure happened more than 30 times this week! Resolving this bug is a high priority.

Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled.

Repository breakdown:

• autoland: 60
• try: 1
• mozilla-central: 7

Platform and build breakdown:

• linux1804-64-qr: 68
  • opt: 68

Table

	a11y-checks-swr
linux1804-64-qr--/opt	68

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1940773&startday=2025-01-20&endday=2025-01-26&tree=all

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ae6cc07c5af6
Skip a11y_checks in browser_permission_prompt_userScripts.js r=rpl

Comment 16

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/ae6cc07c5af6

Comment 17

[Intermittent Failures Robot]

Summary

13 failures in 4566 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 11
• mozilla-central: 2

Platform and build breakdown:

• linux1804-64-qr: 13
  • opt: 13

Table

	a11y-checks-swr
linux1804-64-qr--/opt	13

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1940773&startday=2025-01-27&endday=2025-02-02&tree=all

Comment 18

[James Teh [:Jamie]]

I'm looking into this.

Comment 19

[James Teh [:Jamie]]

Attachment: Bug 1940773: Keep a reference to SimpleTest in AccessibilityUtils.

For some reason I don't understand, window.SpecialPowers.SimpleTest and window.SimpleTest sometimes seem to be cleared during tests (or the global scope changes completely).
Ultimately, we don't control the test scope, so it's best not to rely on it anyway.
Instead, save a reference to SimpleTest in the AccessibilityUtils scope in AccessibilityUtils.init so we can always rely on it being there, since AccessibilityUtils completely controls that scope.
This patch also re-enables the test which was perma failing on CI.

Comment 20

[Rob Wu [:robwu]]

While this fixes the error, there is still the issue of why the todo() was triggered. I described the circumstances leading to this error at https://bugzilla.mozilla.org/show_bug.cgi?id=1940773#c6

Do you know why that is happening? Sounds like a potential issue with a11y tests or an a11y bug in the checkbox implementation.

Comment 21

[James Teh [:Jamie]]

(In reply to Rob Wu [:robwu] from comment #20)

While this fixes the error, there is still the issue of why the todo() was triggered.

Fair. However, I'd suggest we tackle this in a separate bug, since the error is hurting us, where the warning likely isn't, and the fixes are going to be very different.

(In reply to Rob Wu [:robwu] from comment #6)

For some reason, AccessibilityUtils.js thinks that the checkbox does not have a form label, on Linux only (not on macOS)

Note that a11y-checks jobs only run on CI for Linux. There can also be timing issues with Fluent that cause a11y-checks to not be able to examine the label in time, in which case we will pass silently rather than failing. There's not much we can do about that.

at https://searchfox.org/mozilla-central/rev/9f9fa5d2cb72b12ac7e168b7f6ee9820f63291e9/testing/mochitest/tests/SimpleTest/AccessibilityUtils.js#981.

Looking at this briefly, I think this might be happening because the accessibility engine knows how to get the label text for XUL controls which use label="text", but not the label element itself. The label element possibly isn't even in the accessibility tree because check boxes aren't supposed to have children in accessibility terms. So, I think the fix here will probably be to make an exception to avoid this warning in this case in AccessibilityUtils.

Comment 22

[Rob Wu [:robwu]]

(In reply to James Teh [:Jamie] from comment #21)

(In reply to Rob Wu [:robwu] from comment #20)

While this fixes the error, there is still the issue of why the todo() was triggered.

Fair. However, I'd suggest we tackle this in a separate bug, since the error is hurting us, where the warning likely isn't, and the fixes are going to be very different.

I filed bug 1950495 and quoted the relevant parts of the comments here.

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3548a4263a84
Keep a reference to SimpleTest in AccessibilityUtils. r=ayeddi,robwu