Capstone: convert about:privatebrowsing using gandalf's conversion script

RESOLVED FIXED in Firefox 68

Status

()

defect
P3
normal
RESOLVED FIXED
4 months ago
14 days ago

People

(Reporter: Gijs, Assigned: nickcowles9575, Mentored)

Tracking

(Depends on 1 bug, Blocks 1 bug)

Trunk
Firefox 68
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox66 wontfix, firefox67 wontfix, firefox68 fixed)

Details

Attachments

(1 attachment)

(Reporter)

Description

4 months ago
The goal of this bug is twofold:

1) make the script a bit easier to use, and make sure it works.
2) convert about:privatebrowsing.

Gandalf/Zibi wrote some scripts to make it easy to convert XUL-based dtd uses to use fluent, and automatically write a migration script. They live here:

https://github.com/zbraniecki/convert_xul_to_fluent/

To fix this bug:

1) create a clone of the repo in your own github account
2) change convert.py in the toplevel directory to make the following things commandline parameters:
-- the bug id (required)
-- whether to dry-run
-- the target .ftl file path (required)
-- the target repository on which to operate (default to current working directory, currently hardcoded as `../mozilla-unified`)

3) either make the target dtd files another parameter, or offer the user a curses or other interactive interface to select dtd files based on what's in the target .xul file. For this bug, the former is probably sufficient.

4) automatically determine the migration python file name from the bug id and the name of the xul file - probably `bug_{id}_{filename-without.xul}.py` .

5) then run it against https://searchfox.org/mozilla-central/source/browser/components/privatebrowsing/content/aboutPrivateBrowsing.xhtml and its dtd main file (aboutPrivateBrowsing.dtd), potentially also running it against the mobile copy ( https://searchfox.org/mozilla-central/source/mobile/android/chrome/content/aboutPrivateBrowsing.xhtml ) and that DTD file.

5b) *check the result, fix the script as necessary* :-)

6) fix up all the 'before' and 'after' strings to be 1 string each (instead of 3). If you can find a way to make the script "smart" about these cases and do it automatically, that'd be great, but it's not strictly necessary here. Note that you'll need to also fix up the migration recipe for this part.

7) remove all the dtd references from the XHTML file, just leaving `<DOCTYPE html>`. Remove `dir="&locale.dir;"` from the root element.

8) attach a working patch here (include the migration file), and create a PR against gandalf's migration script repo.
Priority: -- → P3
Assignee: nobody → nickcowles9575
Status: NEW → ASSIGNED

Comment 2

2 months ago
Pushed by jwein@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f4094fd9e1df
Capstone: convert about:privatebrowsing using gandalf's conversion script, r=jaws,zbraniecki,flod
No longer blocks: 1484955

Backed out changeset f4094fd9e1df (bug 1517528) for mochitest failures

Backout: https://hg.mozilla.org/integration/autoland/rev/a2e436cd4f21508a895ae7ae4893ea7cf9d7a3ad

Failure push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=f4094fd9e1df1d7d1a43e42ebf4f6cb2c8b3e5c1

Failure log BC:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=229284227&repo=autoland&lineNumber=1678

Failure log en-US:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=229284254&repo=autoland&lineNumber=39984

[task 2019-02-19T20:01:44.884Z] 20:01:44 INFO - TEST-START | browser/base/content/test/static/browser_all_files_referenced.js
[task 2019-02-19T20:01:55.766Z] 20:01:55 INFO - TEST-INFO | started process screentopng
[task 2019-02-19T20:01:56.219Z] 20:01:56 INFO - TEST-INFO | screentopng: exit 0
[task 2019-02-19T20:01:56.219Z] 20:01:56 INFO - Buffered messages logged at 20:01:44
[task 2019-02-19T20:01:56.220Z] 20:01:56 INFO - Entering test bound checkAllTheFiles
[task 2019-02-19T20:01:56.221Z] 20:01:56 INFO - Buffered messages logged at 20:01:55
[task 2019-02-19T20:01:56.221Z] 20:01:56 INFO - indirectly whitelisted file: chrome://marionette/content/test_dialog.dtd used from chrome://marionette/content/test_dialog.xul
[task 2019-02-19T20:01:56.222Z] 20:01:56 INFO - indirectly whitelisted file: chrome://global-platform/locale/intl.properties used from resource://gre/greprefs.js
[task 2019-02-19T20:01:56.223Z] 20:01:56 INFO - indirectly whitelisted file: chrome://marionette/content/test.xul used from chrome://marionette/content/test_anonymous_content.xul
[task 2019-02-19T20:01:56.224Z] 20:01:56 INFO - indirectly whitelisted file: chrome://marionette/content/test_nested_iframe.xul used from chrome://marionette/content/test.xul
[task 2019-02-19T20:01:56.225Z] 20:01:56 INFO - indirectly whitelisted file: chrome://marionette/content/test2.xul used from chrome://marionette/content/test.xul,chrome://marionette/content/test_nested_iframe.xul
[task 2019-02-19T20:01:56.226Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/addonutils.js used from resource://services-sync/engines/addons.js
[task 2019-02-19T20:01:56.227Z] 20:01:56 INFO - indirectly whitelisted file: resource://gre/modules/Promise-backend.js used from resource://gre/modules/Promise.jsm,resource://devtools/shared/worker/loader.js,resource://devtools/shared/Loader.jsm
[task 2019-02-19T20:01:56.228Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/engines/clients.js used from resource://services-sync/service.js
[task 2019-02-19T20:01:56.230Z] 20:01:56 INFO - indirectly whitelisted file: chrome://pippki/content/resetpassword.js used from chrome://pippki/content/resetpassword.xul
[task 2019-02-19T20:01:56.231Z] 20:01:56 INFO - indirectly whitelisted file: chrome://global/content/accessibility/content-script.js used from resource://gre/modules/accessibility/AccessFu.jsm
[task 2019-02-19T20:01:56.232Z] 20:01:56 INFO - indirectly whitelisted file: chrome://global/content/remote-test-ipc.js used from chrome://global/content/test-ipc.xul
[task 2019-02-19T20:01:56.235Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/telemetry.js used from resource://services-sync/bookmark_repair.js,resource://services-sync/service.js
[task 2019-02-19T20:01:56.236Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/stages/enginesync.js used from resource://services-sync/service.js
[task 2019-02-19T20:01:56.237Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/addonsreconciler.js used from resource://services-sync/engines/addons.js
[task 2019-02-19T20:01:56.238Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/collection_validator.js used from resource://services-sync/engines/addons.js,resource://services-sync/engines/forms.js,resource://services-sync/engines/passwords.js
[task 2019-02-19T20:01:56.239Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/policies.js used from resource://services-sync/service.js
[task 2019-02-19T20:01:56.241Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/bookmark_validator.js used from resource://services-sync/engines/bookmarks.js
[task 2019-02-19T20:01:56.242Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/browserid_identity.js used from resource://services-sync/telemetry.js,resource://services-sync/status.js
[task 2019-02-19T20:01:56.244Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-common/tokenserverclient.js used from resource://services-sync/browserid_identity.js
[task 2019-02-19T20:01:56.245Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/status.js used from resource://services-sync/telemetry.js,resource://services-sync/policies.js,resource://services-sync/service.js
[task 2019-02-19T20:01:56.248Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/collection_repair.js used from resource://services-sync/engines/clients.js,resource://services-sync/bookmark_repair.js,resource://services-sync/doctor.js
[task 2019-02-19T20:01:56.249Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/stages/declined.js used from resource://services-sync/service.js
[task 2019-02-19T20:01:56.256Z] 20:01:56 INFO - indirectly whitelisted file: chrome://pippki/content/load_device.js used from chrome://pippki/content/load_device.xul
[task 2019-02-19T20:01:56.257Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/engines.js used from resource://services-sync/engines/clients.js,resource://services-sync/engines/addons.js,resource://services-sync/engines/forms.js,resource://services-sync/engines/history.js,resource://services-sync/engines/tabs.js,resource://services-sync/engines/bookmarks.js,resource://services-sync/engines/extension-storage.js,resource://services-sync/engines/passwords.js,resource://services-sync/engines/prefs.js,resource://services-sync/service.js,resource://formautofill/FormAutofillSync.jsm
[task 2019-02-19T20:01:56.258Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-common/logmanager.js used from resource://services-sync/policies.js
[task 2019-02-19T20:01:56.259Z] 20:01:56 INFO - indirectly whitelisted file: resource://gre/modules/accessibility/EventManager.jsm used from chrome://global/content/accessibility/content-script.js
[task 2019-02-19T20:01:56.259Z] 20:01:56 INFO - indirectly whitelisted file: resource://gre/modules/Http.jsm used from resource://app/modules/translation/GoogleTranslator.jsm,resource://app/modules/translation/YandexTranslator.jsm,resource://app/modules/translation/BingTranslator.jsm
[task 2019-02-19T20:01:56.262Z] 20:01:56 INFO - indirectly whitelisted file: resource://gre/modules/accessibility/Utils.jsm used from chrome://global/content/accessibility/content-script.js,resource://gre/modules/accessibility/Traversal.jsm,resource://gre/modules/accessibility/EventManager.jsm,resource://gre/modules/accessibility/AccessFu.jsm,resource://gre/modules/accessibility/ContentControl.jsm
[task 2019-02-19T20:01:56.263Z] 20:01:56 INFO - indirectly whitelisted file: resource://gre/modules/SyncedBookmarksMirror.jsm used from resource://services-sync/engines/bookmarks.js
[task 2019-02-19T20:01:56.264Z] 20:01:56 INFO - indirectly whitelisted file: resource://gre/modules/accessibility/Constants.jsm used from chrome://global/content/accessibility/content-script.js,resource://gre/modules/accessibility/Traversal.jsm,resource://gre/modules/accessibility/EventManager.jsm,resource://gre/modules/accessibility/Utils.jsm,resource://gre/modules/accessibility/ContentControl.jsm
[task 2019-02-19T20:01:56.265Z] 20:01:56 INFO - indirectly whitelisted file: resource://gre/modules/accessibility/ContentControl.jsm used from chrome://global/content/accessibility/content-script.js
[task 2019-02-19T20:01:56.267Z] 20:01:56 INFO - indirectly whitelisted file: chrome://passwordmgr/content/recipes.json used from resource://gre/greprefs.js
[task 2019-02-19T20:01:56.268Z] 20:01:56 INFO - indirectly whitelisted file: resource://formautofill/FormAutofillSync.jsm used from resource://services-sync/service.js
[task 2019-02-19T20:01:56.269Z] 20:01:56 INFO - indirectly whitelisted file: chrome://global/locale/AccessFu.properties used from resource://gre/modules/accessibility/Utils.jsm
[task 2019-02-19T20:01:56.270Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/bookmark_repair.js used from resource://services-sync/collection_repair.js
[task 2019-02-19T20:01:56.271Z] 20:01:56 INFO - indirectly whitelisted file: resource://services-sync/doctor.js used from resource://services-sync/bookmark_repair.js,resource://services-sync/stages/enginesync.js
[task 2019-02-19T20:01:56.281Z] 20:01:56 INFO - indirectly whitelisted file: resource://gre/modules/accessibility/Traversal.jsm used from resource://gre/modules/accessibility/ContentControl.jsm
[task 2019-02-19T20:01:56.282Z] 20:01:56 INFO - Buffered messages finished
[task 2019-02-19T20:01:56.285Z] 20:01:56 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | there should be no unreferenced files - Got 1, expected 0
[task 2019-02-19T20:01:56.286Z] 20:01:56 INFO - Stack trace:
[task 2019-02-19T20:01:56.287Z] 20:01:56 INFO - chrome://mochikit/content/browser-test.js:test_is:1315
[task 2019-02-19T20:01:56.287Z] 20:01:56 INFO - chrome://mochitests/content/browser/browser/base/content/test/static/browser_all_files_referenced.js:checkAllTheFiles:784
[task 2019-02-19T20:01:56.287Z] 20:01:56 INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
[task 2019-02-19T20:01:56.289Z] 20:01:56 INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1134
[task 2019-02-19T20:01:56.289Z] 20:01:56 INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
[task 2019-02-19T20:01:56.289Z] 20:01:56 INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
[task 2019-02-19T20:01:56.290Z] 20:01:56 INFO - Not taking screenshot here: see the one that was previously logged
[task 2019-02-19T20:01:56.291Z] 20:01:56 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | unreferenced file: resource://app/localization/en-US/browser/aboutPrivateBrowsing.ftl -
[task 2019-02-19T20:01:56.291Z] 20:01:56 INFO - Stack trace:
[task 2019-02-19T20:01:56.292Z] 20:01:56 INFO - chrome://mochikit/content/browser-test.js:test_ok:1304
[task 2019-02-19T20:01:56.292Z] 20:01:56 INFO - chrome://mochitests/content/browser/browser/base/content/test/static/browser_all_files_referenced.js:checkAllTheFiles:788
[task 2019-02-19T20:01:56.293Z] 20:01:56 INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
[task 2019-02-19T20:01:56.294Z] 20:01:56 INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1134
[task 2019-02-19T20:01:56.295Z] 20:01:56 INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
[task 2019-02-19T20:01:56.296Z] 20:01:56 INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
[task 2019-02-19T20:01:56.297Z] 20:01:56 INFO - ignored unused whitelist entry: resource://app/blocklist.xml
[task 2019-02-19T20:01:56.298Z] 20:01:56 INFO - ignored unused whitelist entry: resource://gre/gmp-clearkey/0.1/manifest.json
[task 2019-02-19T20:01:56.299Z] 20:01:56 INFO - ignored unused whitelist entry: resource://gre/res/test.properties
[task 2019-02-19T20:01:56.300Z] 20:01:56 INFO - missing file: resource://gre/components/nsAsyncShutdown.js
[task 2019-02-19T20:01:56.301Z] 20:01:56 INFO - missing file: resource://gre/modules/commonjs/toolkit/loader.js
[task 2019-02-19T20:01:56.302Z] 20:01:56 INFO - missing file: resource://app/localization/en-US/security/certificates/deviceManager.ftl referenced from chrome://pippki/content/load_device.xul,chrome://pippki/content/device_manager.xul,chrome://browser/content/preferences/in-content/preferences.xul

Flags: needinfo?(nickcowles9575)
Attachment #9042691 - Attachment description: Bug 1517528 - Capstone: convert about:privatebrowsing using gandalf's conversion script, r?Gijs,jaws → Bug 1517528 - Capstone: convert about:privatebrowsing using gandalf's conversion script, r?Gijs
Attachment #9042691 - Attachment description: Bug 1517528 - Capstone: convert about:privatebrowsing using gandalf's conversion script, r?Gijs → Bug 1517528 - Capstone: convert about:privatebrowsing using gandalf's conversion script, r?Gijs,jaws

Comment 4

2 months ago
Pushed by gijskruitbosch@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/40ddf7d53eaa
Capstone: convert about:privatebrowsing using gandalf's conversion script, r=Gijs,jaws,zbraniecki,flod

Sorry, this had to be backed out for flake8 failures:

https://hg.mozilla.org/integration/autoland/rev/234f0dde00661d41a6641e950ded8bd93aabf9a9

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&group_state=expanded&revision=40ddf7d53eaaf7ea9b3bf134ad0030713dd0e1a2
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=231217756&repo=autoland
TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/python/l10n/convert_xul_to_fluent/convert.py:2:38 | trailing whitespace (W291)
etc. See the linked log for failures.

The |mach lint| command can check for those failures.

Duplicate of this bug: 1486916
(Assignee)

Updated

a month ago
Depends on: 1534310

Comment 7

a month ago
Pushed by jwein@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f73f629e8cca
Capstone: convert about:privatebrowsing using gandalf's conversion script, r=Gijs,jaws,zbraniecki,flod

Backed out changeset f73f629e8cca (bug 1517528) on request from jaws, CLOSED TREE.

Backout link: https://hg.mozilla.org/integration/autoland/rev/6d98527df96dc3c8d3bc30c2847b8675c4e46f44

Flags: needinfo?(jaws)

Comment 9

28 days ago
Pushed by jwein@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e34eb1bed1a9
Capstone: convert about:privatebrowsing using gandalf's conversion script, r=Gijs,jaws,zbraniecki,flod

Comment 10

28 days ago
bugherder
Status: ASSIGNED → RESOLVED
Last Resolved: 28 days ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 68
Flags: needinfo?(jaws)
Flags: needinfo?(nickcowles9575)
You need to log in before you can comment on or make changes to this bug.