Bugzilla

We discussed this, and decided to make the version stricter after all. If needed we can relax the requirement later, but the other way around is not feasible.

We're going to require a strict format in MV3, and warn when used in MV2.

The implementation is straightforward, at https://searchfox.org/mozilla-central/rev/f0ffbdcf82000a8ecab13954f6618797e37c6173/toolkit/components/extensions/schemas/manifest.json#54-57:

- Introduce validation of the version format.
- Treat it as a warning in MV2 (e.g. `"onError": "warn"`) (bug 1793925).
- Treat it as an error in MV3 (same validation, without `"onError": "warn"`) (this bug?).

To assess the impact of this change, I ran some queries to see how many extensions would be considered invalid with the new format. 
Note that the queries did not account for users who have installed two of the affected add-ons, these would be over-counted in the results. I expect that impact to be negligible. The results are as follows:

- Total users of add-ons that have had at least one invalid version: https://sql.telemetry.mozilla.org/queries/87982/source
  - 24M users from 3257 extensions
  - 3M users from 186 dictionaries
  - 34M users from 443 langpacks
  - 14k users from 65 static themes.
- Total users of add-ons whose **latest** version is an invalid version: https://sql.telemetry.mozilla.org/queries/87983/source
  - 206k users from 130 extensions
  - 1.8M users from 142 dictionaries
  - 34M users from 105 langpacks
  - 0.5k users from 10 static themes.
- Breakdown of individual extensions whose latest version is invalid: https://sql.telemetry.mozilla.org/queries/87984/source
 - e.g. `0.5rc2` from RecEx https://addons.mozilla.org/en-US/firefox/addon/cookie-quick-manager/
 - e.g. `0.4.5.1webext` from dictionary https://addons.mozilla.org/en-US/firefox/addon/russian-spellchecking-dic-3703/ (suffix "webext" seems to be common)
 - e.g. `1.1.4buildid20220803.111909` from privileged extension https://addons.mozilla.org/en-US/firefox/addon/firefox-translations/
 - e.g. `105.0.3buildid20221007.134813` from langpack https://addons.mozilla.org/en-US/firefox/addon/english-gb-language-pack/
- Version numbers whose major version (i.e. leading component of the version) exceeds 65335: https://sql.telemetry.mozilla.org/queries/87985/source
  - 1 extension with version `20220804.0wot` at https://addons.mozilla.org/en-US/firefox/addon/wot-safe-browsing-tool/
  - 13 dictionaries with 13k users in total, with the format ranging in the format `YYYYMMDD.<digit>webext`. E.g. 20180701.1webext and 20110323.2webext.
  - The significance of this is that IF we were to strictly enforce version numbers, that the extensions in this list would be unable to upgrade because Firefox does now downgrade extension versions.

To determine validity, I used the following regexps:
- Invalid version: `^([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])(\.([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])){0,3}$`. While this ensures that the version number is between 0 and 65335, inclusive, it may accept ambiguous version numbers (e.g. 0 vs 00, 0.01 vs 0.1 or even 0.01.2.3 vs 0.1.2.3).
- Major version too high: `^([0-9]{6}|[7-9][0-9]{4}|6[6-9][0-9]{3}|65[4-9][0-9]{2}|653[4-9][0-9]|6533[6-9])`

Follow-ups:
- We should determine the desired behavior with version parts starting with 0. E.g. 0.1 is obviously valid. But how about 0.01 vs 0.001? or 0001 vs 01?
- What to do with the few add-ons that have a too large major version number? 1 extension and 13 (auto-generated?) dictionaries.
- Fix langpack generation script (e.g. like done in bug 1732676) and ensure that the privileged extension pipeline generates valid versions. This should cover 36M users.
- The dictionaries (1.8M users) seem to have been auto-generated and/or nor updated in a while. We should look into fixing the version. Note that there may be dictionaries that we don't know about, because currently dictionaries are not required to be signed. This fix-up effort could be considered along with enforcing dictionary signing in bug 1753276.
- If the remaining 140 extensions/themes (~200k users) were to update to a valid version, then we could enforce the new version format on AMO even for MV2 extensions, ahead of requiring it in Firefox.

We discussed this, and decided to make the version stricter after all. If needed we can relax the requirement later, but the other way around is not feasible.

We're going to require a strict format in MV3, and warn when used in MV2.

The implementation is straightforward, at https://searchfox.org/mozilla-central/rev/f0ffbdcf82000a8ecab13954f6618797e37c6173/toolkit/components/extensions/schemas/manifest.json#54-57:

- Introduce validation of the version format.
- Treat it as a warning in MV2 (e.g. `"onError": "warn"`) (bug 1793925).
- Treat it as an error in MV3 (same validation, without `"onError": "warn"`) (this bug?).

To assess the impact of this change, I ran some queries to see how many extensions would be considered invalid with the new format. 
Note that the queries did not account for users who have installed two of the affected add-ons, these would be over-counted in the results. I expect that impact to be negligible. The results are as follows:

- Total users of add-ons that have had at least one invalid version: https://sql.telemetry.mozilla.org/queries/87982/source
  - 24M users from 3257 extensions
  - 3M users from 186 dictionaries
  - 34M users from 443 langpacks
  - 14k users from 65 static themes.
- Total users of add-ons whose **latest** version is an invalid version: https://sql.telemetry.mozilla.org/queries/87983/source
  - 206k users from 130 extensions
  - 1.8M users from 142 dictionaries
  - 34M users from 105 langpacks
  - 0.5k users from 10 static themes.
- Breakdown of individual extensions whose latest version is invalid: https://sql.telemetry.mozilla.org/queries/87984/source
  - e.g. `0.5rc2` from RecEx https://addons.mozilla.org/en-US/firefox/addon/cookie-quick-manager/
  - e.g. `0.4.5.1webext` from dictionary https://addons.mozilla.org/en-US/firefox/addon/russian-spellchecking-dic-3703/ (suffix "webext" seems to be common)
  - e.g. `1.1.4buildid20220803.111909` from privileged extension https://addons.mozilla.org/en-US/firefox/addon/firefox-translations/
  - e.g. `105.0.3buildid20221007.134813` from langpack https://addons.mozilla.org/en-US/firefox/addon/english-gb-language-pack/
- Version numbers whose major version (i.e. leading component of the version) exceeds 65335: https://sql.telemetry.mozilla.org/queries/87985/source
  - 1 extension with version `20220804.0wot` at https://addons.mozilla.org/en-US/firefox/addon/wot-safe-browsing-tool/
  - 13 dictionaries with 13k users in total, with the format ranging in the format `YYYYMMDD.<digit>webext`. E.g. 20180701.1webext and 20110323.2webext.
  - The significance of this is that IF we were to strictly enforce version numbers, that the extensions in this list would be unable to upgrade because Firefox does now downgrade extension versions.

To determine validity, I used the following regexps:
- Invalid version: `^([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])(\.([0-9]{1,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])){0,3}$`. While this ensures that the version number is between 0 and 65335, inclusive, it may accept ambiguous version numbers (e.g. 0 vs 00, 0.01 vs 0.1 or even 0.01.2.3 vs 0.1.2.3).
- Major version too high: `^([0-9]{6}|[7-9][0-9]{4}|6[6-9][0-9]{3}|65[4-9][0-9]{2}|653[4-9][0-9]|6533[6-9])`

Follow-ups:
- We should determine the desired behavior with version parts starting with 0. E.g. 0.1 is obviously valid. But how about 0.01 vs 0.001? or 0001 vs 01?
- What to do with the few add-ons that have a too large major version number? 1 extension and 13 (auto-generated?) dictionaries.
- Fix langpack generation script (e.g. like done in bug 1732676) and ensure that the privileged extension pipeline generates valid versions. This should cover 36M users.
- The dictionaries (1.8M users) seem to have been auto-generated and/or nor updated in a while. We should look into fixing the version. Note that there may be dictionaries that we don't know about, because currently dictionaries are not required to be signed. This fix-up effort could be considered along with enforcing dictionary signing in bug 1753276.
- If the remaining 140 extensions/themes (~200k users) were to update to a valid version, then we could enforce the new version format on AMO even for MV2 extensions, ahead of requiring it in Firefox.