Closed Bug 1494926 Opened 6 years ago Closed 6 years ago

Prevent old JAWS users from updating past Firefox 60 ESR

Categories

(Core :: Disability Access APIs, enhancement, P1)

Product:
Core
Component:
Disability Access APIs
Platform:
All
Windows
Type:
enhancement

Tracking

()

Status:
VERIFIED FIXED
Tracking Flags:
Tracking Status
firefox-esr60 69+ verified

People

(Reporter: Jamie, Assigned: yzen)

References

Details

Attachments

(3 files, 2 obsolete files)

Bug 1494926 - updating jaws-esr add-on with newer localization strings and a way for accessibility users to re-enable e10s. r=rhelmer
47 bytes, text/x-phabricator-request
RyanVM
: approval-mozilla-esr60+
Details | Review
Bug 1494926 - add the JAWS update URL token to update URL. r=rstrong
47 bytes, text/x-phabricator-request
RyanVM
: approval-mozilla-esr60+
Details | Review
111.png
197.62 KB, image/png
Details
Bug 1489605 shipped a system add-on for 60 ESR to disable e10s for users of JAWS versions before 2018 due to incompatibility. Non-e10s is not a supported configuration in Firefox 60, though it still seems to work well enough. With each new release, the risk of non-e10s breaking increases. While we wanted to correct the oversight of automatically (and without warning) updating old JAWS users to a broken state, we cannot support these users in future versions of Firefox. (Note also that the JAWS vendor officially considers JAWS 18 end of life as of next month.) Thus, we should prevent users of old JAWS from being updated past Firefox 60 ESR, just as we prevented Windows XP users from being updated past 52 ESR. The intuitive approach is to ship a change in a 60 ESR update which provides the update server with the old JAWS flag and then have the update server block updates accordingly. The major problem is that a user might run Firefox at some point with another screen reader (e.g. NVDA) or no screen reader at all, at which point they would be updated and broken. The absence of old JAWS in a given session does not indicate that it is safe to update. We must only allow updates once the presence of new JAWS is detected, as a user probably won't use two versions of JAWS in real usage. Because of this problem, I think we'll instead need to disable updates using the system add-on, if possible. This way, we can disable them if old JAWS is detected and re-enable them if new JAWS is detected.
See Also: → 1494929
This feels like a decision we don't need to make for a long time. In the mean time, are we collecting the data we need to make an informed choice here?
tracking-firefox-esr60: --- → ?
Yes, we have telemetry for old JAWS users on ESR 60, including data about those for whom e10s was disabled by the system add-on.

Given that we're into the Nightly 68 development cycle now, we should probably circle back here and figure out what we need to do for ESR60 and possibly ESR68 while there's still time to create and land the patches :)

Flags: needinfo?(jteh)

Marking this for 68 as well since it'll be a major ESR version.

status-firefox68: --- → affected
tracking-firefox68: --- → ?
Priority: P2 → P1

Yura is working on this. See the doc linked in comment 5 for details.

Assignee: nobody → yzenevich
Status: NEW → ASSIGNED
Flags: needinfo?(jteh)

Yura can you provide a status update on this work?

Flags: needinfo?(yzenevich)

We are waiting of localization until about mid June, by then I'm gonna make sure that the add-on updates are working locally (e.g. manual testing) before we can bundle/test/deploy.

Flags: needinfo?(yzenevich)
Attached file GitHub Pull Request (obsolete) —
Attached file jaws-esr@mozilla.org-v2.0.xpi (obsolete) —

This is an unsigned xpi that contains l10n bundles and updated code.

Attachment #9075755 - Attachment is obsolete: true
Attachment #9075749 - Attachment is obsolete: true

A quick update about this bug:

  1. JAWS update URL token was removed in Firefox 59 (see bug 1413727) which means, if I understand correctly, there is no way of blocking OLDJAWS users from updating to 68.
    This requires to re-land a patch similar to bug 1402376 with some modifications that will take into account not only if OLDJAWS is currently running but also how long it has been since the last time OLDJAWS screen reader was run.
    This modification will not require any changes to SYSTEM_CAPABILITIES since it already understands how to parse JAWS token (see bug 1402411).
    This modification however will require a watershed dot release release (or be included into a dot release) on 60ESR that happens before the switch to 68ESR happens.

  2. When we shipped original add-on last year via balrog, we also consequently landed it into tree of Firefox60ESR (see bug 1489605).
    Given that we would already need a dot (watershed) release for changes in (1), if I understand correctly, we can simply land changes to the add-on in tree without requiring a one off system add-on approach.

This will probably also simplify things somewhat for QA'ing the add-on. Flagging Ben and Rob to confirm.

Flags: needinfo?(rhelmer)
Flags: needinfo?(bhearsum)

(In reply to Yura Zenevich [:yzen] from comment #11)

A quick update about this bug:

  1. JAWS update URL token was removed in Firefox 59 (see bug 1413727) which means, if I understand correctly, there is no way of blocking OLDJAWS users from updating to 68.
    This requires to re-land a patch similar to bug 1402376 with some modifications that will take into account not only if OLDJAWS is currently running but also how long it has been since the last time OLDJAWS screen reader was run.
    This modification will not require any changes to SYSTEM_CAPABILITIES since it already understands how to parse JAWS token (see bug 1402411).

As long as the client only sends JAWS:1 when it shouldn't update, we're good on the Balrog side.

This modification however will require a watershed dot release release (or be included into a dot release) on 60ESR that happens before the switch to 68ESR happens.

I think we should be clear about the cost benefit of having a watershed to support this. Watershed releases typically slow our update rate (as Robert often talks about), so I'd like us to be sure we're OK with doing this for whatever number of JAWS users this will affect.

Flags: needinfo?(bhearsum)
Flags: needinfo?(rhelmer)

Adding Robert Strong, because I think we got the wrong Robert :).

(In reply to Yura Zenevich [:yzen] from comment #11)

  1. When we shipped original add-on last year via balrog, we also consequently landed it into tree of Firefox60ESR (see bug 1489605).
    Given that we would already need a dot (watershed) release for changes in (1), if I understand correctly, we can simply land changes to the add-on in tree without requiring a one off system add-on approach.

Well, for my part the above sgtm :)

I'm fine with this on 60ESR. What is the percentage of clients using old JAWS on ESR?

Flags: needinfo?(yzenevich)

It's around 8% of all of our screen reader users.
For reference if necessary these are accessibility users in general: https://sql.telemetry.mozilla.org/dashboard/parent-a11y-consumers
And this is for JAWS on ESR https://sql.telemetry.mozilla.org/queries/59078#152975
As for JAWS/OLDJAWS on ESR only it's about 30% each of all screen reader users on ESR: https://sql.telemetry.mozilla.org/queries/63567

Flags: needinfo?(yzenevich)

Do you have percentage of old JAWS on ESR out of all ESR clients?

(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #19)

Do you have percentage of old JAWS on ESR out of all ESR clients?

From what I can tell there are about 5.6 million daily users on ESR on Windows and about a thousand OLDJAWS users. Which makes it roughly 0.02%.

Thanks for the data!

Send a PI request for the above commits for when they land in ESR60

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9fd5b83be775d1415f54417b9be92b127b25e6cd <- build with both patches applied (though for some reason it seems like the older version, or perhaps, what's been deployed via balrog, is still somehow lingering around).

(In reply to Yura Zenevich [:yzen] from comment #23)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9fd5b83be775d1415f54417b9be92b127b25e6cd <- build with both patches applied (though for some reason it seems like the older version, or perhaps, what's been deployed via balrog, is still somehow lingering around).

What do you mean? I'm not sure how this try push and any Balrog code or updates that we've pushed connect.

Comment on attachment 9077302 [details]
Bug 1494926 - updating jaws-esr add-on with newer localization strings and a way for accessibility users to re-enable e10s. r=rhelmer

ESR Uplift Approval Request

  • If this is not a sec:{high,crit} bug, please state case for ESR consideration: This is to support old JAWS screen reader users that must remain on esr60 to not end up with a broken browser + screen reader configuration. This updated add-on also re-enabled e10s for users who previously used old JAWS but now updated and lets them move to ESR68.
  • User impact if declined: Users who are using old JAWS screen reader will be updated to ESR68 and will, highly likely, end up with a broken configuration.
  • Fix Landed on Version:
  • Risk to taking this patch: Medium
  • Why is the change risky/not risky? (and alternatives if risky): This is an update to an add-on that already shipped in ESR60. It adds some complexity in regards to re-enabling e10s for users that do not user old JAWS any more. The add-on also checks and updates a time stamp for the last use of old JAWS if necessary once a week.
  • String or UUID changes made by this patch:
Attachment #9077302 - Flags: approval-mozilla-esr60?
Attachment #9077303 - Flags: approval-mozilla-esr60?

(In reply to bhearsum@mozilla.com (:bhearsum) from comment #24)

(In reply to Yura Zenevich [:yzen] from comment #23)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9fd5b83be775d1415f54417b9be92b127b25e6cd <- build with both patches applied (though for some reason it seems like the older version, or perhaps, what's been deployed via balrog, is still somehow lingering around).

What do you mean? I'm not sure how this try push and any Balrog code or updates that we've pushed connect.

sorry I found a problem where the profile had an old add-on in place.

Hi Ben, when the patches land in the esr60 branch, I wanted to confirm that on the balrog side, the behaviour will be as expected around the time the updated to the new ESR will be happening - that the users with JAWS:1 will not be updated to ESR68. Thanks

Flags: needinfo?(bhearsum)

(In reply to Yura Zenevich [:yzen] from comment #27)

Hi Ben, when the patches land in the esr60 branch, I wanted to confirm that on the balrog side, the behaviour will be as expected around the time the updated to the new ESR will be happening - that the users with JAWS:1 will not be updated to ESR68. Thanks

This will be done by whomever is on releaseduty as part of the ESR68 release work. It looks like that's Tom Prince, so I'm cc'ing him. Tom - let me know if you need any additional background on this.

Flags: needinfo?(bhearsum)

Comment on attachment 9077302 [details]
Bug 1494926 - updating jaws-esr add-on with newer localization strings and a way for accessibility users to re-enable e10s. r=rhelmer

Updates the ESR60 jaws-esr addon to block automatic updates to ESR68 for users of older JAWS versions. Also re-enables e10s and allows users to update when they've updated their JAWS version. Per comment 20, this is expected on only affect a very small set of users, but those users will be left in a basically unusable state if allowed to update. Approved for 60.9esr.

Attachment #9077302 - Flags: approval-mozilla-esr60? → approval-mozilla-esr60+
Attachment #9077303 - Flags: approval-mozilla-esr60? → approval-mozilla-esr60+

This definitely needs QA testing before we ship. 60.9esr is the last scheduled release of the ESR60 line and we don't want to be in a position where there's an issue we're not in a position to ship a fix for anymore.

Flags: qe-verify+

Backed out for failing a11y's accessible/tests/mochitest/actions/test_anchors.html after asserting:

https://hg.mozilla.org/releases/mozilla-esr60/rev/52543e3cf9d0c98ecfbe805bd6d95964e97e4fe6

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr60&group_state=expanded&selectedJob=259618075&resultStatus=testfailed%2Cbusted%2Cexception&revision=9bcfd93bc1467b59af55e84bfdcdf456d438c1c0
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=259618075&repo=mozilla-esr60

Assertion failure: gConsumers (No creation after shutdown), at z:/build/build/src/accessible/base/nsAccessibilityService.cpp:1003
ERROR TEST-UNEXPECTED-FAIL | accessible/tests/mochitest/actions/test_anchors.html | application terminated with exit code 1
PROCESS-CRASH | accessible/tests/mochitest/actions/test_anchors.html | application crashed [@ nsAccessibilityService::CreateAccessible(nsINode *,mozilla::a11y::Accessible *,bool *)]

Status: RESOLVED → REOPENED
status-firefox70: fixed → ---
Flags: needinfo?(yzenevich)
Resolution: FIXED → ---
Target Milestone: mozilla70 → ---

Hi, I believe the failure is now fixed and the phabricator revisions are now updated. Could we try landing it again?

Flags: needinfo?(aryx.bugmail)

This landed:
https://hg.mozilla.org/releases/mozilla-esr60/rev/91c16abf36fa5ad6f63e53b0693ccf75ed9a15a0
https://hg.mozilla.org/releases/mozilla-esr60/rev/8f93072480db437b2d5754a6c40d531cefcf13f0

And got backed out for frequently failing browser-chrome with 'Waited too long for window with dialog to focus' on Linux Nightly:
https://hg.mozilla.org/releases/mozilla-esr60/rev/3ae7c5b3f6e7f0fdf0cd7a92086484bafa6f9a81

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr60&selectedJob=260592596&resultStatus=testfailed%2Cbusted%2Cexception&revision=8f93072480db437b2d5754a6c40d531cefcf13f0
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=260592596&repo=mozilla-esr60

14:07:11 ERROR - GECKO(5866) | TEST-UNEXPECTED-FAIL | unknown test url | Waited too long for window with dialog to focus
4962 14:07:11 INFO - TEST-UNEXPECTED-FAIL | toolkit/components/startup/tests/browser/browser_bug511456.js | Should have seen a prompt dialog -

Flags: needinfo?(aryx.bugmail) → needinfo?(yzenevich)

Verified with the taskcluster builds for 60.9esr, with JAWS 18.0.5038 64-bit English - May 2018 ---- Windows 10.
The prompt/restart, timmers(custom_build provided by Yura) and localised build(FR) are working as intended.

status-firefox-esr60: fixed → verified

@Aryx can we update the status of the bug as well since code is relanded?

Flags: needinfo?(aryx.bugmail)
Status: REOPENED → RESOLVED
Closed: 6 years ago → 6 years ago
status-firefox-esr68: affected → ---
tracking-firefox-esr68: 69+ → ---
Flags: needinfo?(aryx.bugmail)
Resolution: --- → FIXED

Updating remaining flags.

Status: RESOLVED → VERIFIED
Flags: qe-verify+

I heard that :rail was pinged on Monday for Release Engineering support - for testing, IIUC?
I'm on releaseduty for RelEng at the moment, so perhaps I can help here. Let me know what's needed :)

Flags: needinfo?(yzenevich)

(In reply to Mitchell Hentges [:mhentges] from comment #42)

I heard that :rail was pinged on Monday for Release Engineering support - for testing, IIUC?
I'm on releaseduty for RelEng at the moment, so perhaps I can help here. Let me know what's needed :)

Hi Mitchell, I was asking Rail how feasible would it be to make the 60.9 release a watershed one. We would love not to break as many screen reader users as possible.

Thanks

Flags: needinfo?(yzenevich) → needinfo?(mhentges)

Verified with the archive build for 60.9esr, with JAWS 18.0.5038 64-bit English , under Windows 10, and did some exploratory too on random sites, all seems to work well.

Now that the mergeday is complete, I'll set this up at some point this week

Flags: needinfo?(mhentges)

Balrog rules added to esr-localtest-next, asking RelMan for QE assistance

As a status update for this issue:
On the QA side of things, we've started working with Mitchell, and we've sorted the details/STR for verification on slack.
We're currently waiting for the server-side changes to check for the "simulated" update to 68.

Stumbled today over the reason why the update (probably) was not going through for this:

  • with JAWS running the update did not start;
  • with JAWS closed the update was performed without any issue.

After talking to yzen, I've learned that the Firefox update won't happen until JAWS updates and a week passes (it has to be a week since an old, incompatible version of JAWS was used).

So, we should be able to continue testing this on this Thursday.

Attached image 111.png —

Apparently there are still some issues that might need to be sorted out with this.

  • JAWS was with the latest version installed and unchanged;
  • when opening the ESR client the displayed notification popped up.
Flags: needinfo?(mhentges)

Good news, after the 2019 JAWS installation the update went through.
With "esr-localtest-next":

  • Tried with the previous files and the update went from the first attempt.
  • After checking with another fresh instance of 60.0.1esr, with the first restart it updated to the expected ESR versions, 60.9 and after that to 68.1.0.

I'm glad the update went through! The upgrade path looks sane, too (60.0.1esr => 60.9esr => 68.1.0esr). 👍
I talked to :cfogel directly, and we're going to check the behaviour of JAWS 2020 as well, while we're here.
If/when that's working as expected, I'll add these balrog rules to the real esr channel.

Flags: needinfo?(mhentges)

Same case for JAWS 2020:
With an installation of 60.0esr the update path was as expected once again(60.0esr => 60.9esr => 68.1.0esr).

I'll update the real esr release channel in balrog with my same rules.
Once ESR 68.2.0 is released, we can re-verify the update logic

Flags: needinfo?(mhentges)

I've signed off on the ESR rule changes in Balrog.

Flags: needinfo?(mhentges)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: