1667276 - (backgroundtasks) Add a Gecko background task mode and `--backgroundtask` flag to allow running short-lived processes that perform maintenance

Status: RESOLVED FIXED

(Toolkit :: Application Update, enhancement)

Description

[Nick Alexander :nalexander [he/him]]

This is a ticket for experimentation. We'd like to support something like firefox --backgroundtask update that would pump the UpdateService loop from the command line. Here's a place to hang work in progress.

Comment 1

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - WIP on a `--backgroundtask ...` mode for running things at the command line.

Obvious TODOs:

• Use the static BackgroundTasks class rather than hardcoding the
  changes to the manifest parser and to the static component
  registration, so that regular Firefox still works.

• Make the static component registration futzing data driven.

Depends on D91372

Comment 2

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - TO FOLD - Import a JS module rather than instantiating an XPCOM interface.

Depends on D91373

Comment 3

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - TO FOLD - Handle async, allow a return code.

Depends on D94209

Comment 4

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - Actually check for updates.

Depends on D94210

Comment 5

[Dave Townsend [:mossop]]

Attachment: Bug 1667276: PoC for extracting data from the default profile.

Adds a couple of useful APIs for background tasks. The main one is for safely
locking the profile so that data can be extracted and then unlocking it. A
second API allows for parsing the prefs file from the profile.

Comment 6

[Dave Townsend [:mossop]]

Attachment: Bug 1667276 - Pre: Disable the startup cache when running a background task. r?sg!

Depends on D95060

Comment 7

[Dave Townsend [:mossop]]

Attachment: Bug 1667276: Load a custom prefs file when running a background task.

Depends on D95198

Comment 8

[Dave Townsend [:mossop]]

Attachment: Bug 1667276: Disable persistent cookie storage when running background tasks.

Depends on D95199

Comment 9

[Dave Townsend [:mossop]]

Attachment: Bug 1667276: Disable various caches.

Depends on D95200

Comment 10

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - WIP on a test, Telemetry, and devtools support.

Depends on D91373

Comment 11

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - Pre: Disable extension manager in background tasks mode. r?Mossop

Depends on D95200

Comment 12

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - Part 3: Load a custom prefs file when running a background task. r?Mossop!,KrisWright!

Depends on D97509

Comment 13

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - Part 1: Wire background task mode into Gecko startup. r?Mossop

Depends on D97510

Comment 14

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - Part 2: Add BackgroundTasksManager to invoke task defined in JS. r?Mossop

Depends on D97511

Comment 15

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1667276 - Post: Add test limiting the XPCOM graph of the no-op wait background task. r?Mossop

This establishes a high water mark for code loaded (even after a short
delay) by a background task that does nothing.

Code loaded here means:

1. Chrome JSMs imported using ChromeUtils.import;

2. XPCOM services, generally long-lived, loaded using do_getService
   or Services.* or an equivalent;

3. XPCOM components defined in JavaScript and loaded via
   chrome.manifest entries.

At this time background tasks do not load any of category 3. The
distinction is made because they are reported separately by Gecko.

Depends on D97512

Comment 16

[Phabricator Automation]

Comment on attachment 9184605 [details]
Bug 1667276: Disable persistent cookie storage when running background tasks.

Revision D95200 was moved to bug 1675829. Setting attachment 9184605 [details] to obsolete.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ab6f830f6e75
Pre: Disable the startup cache when running a background task. r=dthayer
https://hg.mozilla.org/integration/autoland/rev/74580a0f2633
Pre: Disable extension manager in background tasks mode. r=mossop
https://hg.mozilla.org/integration/autoland/rev/cf52687c4433
Part 1: Wire background task mode into Gecko startup. r=mossop
https://hg.mozilla.org/integration/autoland/rev/60c2f2dbc676
Part 2: Add BackgroundTasksManager to invoke task defined in JS. r=mossop
https://hg.mozilla.org/integration/autoland/rev/38c20196aabc
Part 3: Load a custom prefs file when running a background task. r=mossop,KrisWright
https://hg.mozilla.org/integration/autoland/rev/f970ef0897cd
Post: Add test limiting the XPCOM graph of the no-op wait background task. r=mossop

Comment 18

[Natalia Csoregi [:nataliaCs]]

Backed out for causing browser_all_files_referenced.js failures.

backout: https://hg.mozilla.org/integration/autoland/rev/06ee1ff0214b90a8b2689141d33e9d756b2351ae

push: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&test_paths=browser%2Fbase%2Fcontent%2Ftest%2Fstatic%2Fbrowser.ini&revision=f1a65c9b3ca29e352a6fa427ec811b816bf328c2&selectedTaskRun=aVIAbSqGSQ2qYD8IKfWdUQ.1

failure log: https://treeherder.mozilla.org/logviewer?job_id=327785851&repo=autoland&lineNumber=2582

[task 2021-01-26T03:44:56.643Z] 03:44:56 INFO - indirectly whitelisted file: chrome://fxr/content/prefs.js used from chrome://fxr/content/prefs.html
[task 2021-01-26T03:44:56.644Z] 03:44:56 INFO - Buffered messages finished
[task 2021-01-26T03:44:56.648Z] 03:44:56 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | there should be no unreferenced files - Got 4, expected +0
[task 2021-01-26T03:44:56.648Z] 03:44:56 INFO - Stack trace:
[task 2021-01-26T03:44:56.648Z] 03:44:56 INFO - chrome://mochikit/content/browser-test.js:test_is:1351
[task 2021-01-26T03:44:56.649Z] 03:44:56 INFO - chrome://mochitests/content/browser/browser/base/content/test/static/browser_all_files_referenced.js:checkAllTheFiles:948
[task 2021-01-26T03:44:56.649Z] 03:44:56 INFO - Not taking screenshot here: see the one that was previously logged
[task 2021-01-26T03:44:56.650Z] 03:44:56 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/static/browser_all_files_referenced.js | unreferenced file: resource://gre/defaults/backgroundtasks/backgroundtasks.js -
[task 2021-01-26T03:44:56.650Z] 03:44:56 INFO - Stack trace:
[task 2021-01-26T03:44:56.651Z] 03:44:56 INFO - chrome://mochikit/content/browser-test.js:test_ok:1323
[task 2021-01-26T03:44:56.651Z] 03:44:56 INFO - chrome://mochitests/content/browser/browser/base/content/test/static/browser_all_files_referenced.js:checkAllTheFiles:952
[task 2021-01-26T03:44:56.652Z] 03:44:56 INFO - Not taking screenshot here: see the one that was previously logged

Also failing: https://treeherder.mozilla.org/logviewer?job_id=327785366&repo=autoland&lineNumber=5074

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c673fff5bd85
Pre: Disable the startup cache when running a background task. r=dthayer
https://hg.mozilla.org/integration/autoland/rev/1dcb2d1be264
Pre: Disable extension manager in background tasks mode. r=mossop
https://hg.mozilla.org/integration/autoland/rev/62488ec634f9
Part 1: Wire background task mode into Gecko startup. r=mossop
https://hg.mozilla.org/integration/autoland/rev/8d6f10d83c6b
Part 2: Add BackgroundTasksManager to invoke task defined in JS. r=mossop
https://hg.mozilla.org/integration/autoland/rev/1c8d51d2c90f
Part 3: Load a custom prefs file when running a background task. r=mossop,KrisWright
https://hg.mozilla.org/integration/autoland/rev/9ab334e527a5
Post: Add test limiting the XPCOM graph of the no-op wait background task. r=mossop

Comment 20

[Atila Butkovits]

Backed out for causing failure on test_crash_backgroundtask_moz_crash.js.

Backout link: https://hg.mozilla.org/integration/autoland/rev/e379907d0452a34155518062e838b465f5e4a37c

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&searchStr=linux%2C18.04%2Cx64%2Copt%2Cxpcshell%2Ctests%2Ctest-linux1804-64%2Fopt-xpcshell-e10s%2Cx2&revision=f06504e3219f5a31ce538829376eeb53be317715&selectedTaskRun=SF47dsaJR4uEMyYymhI9nQ.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=328010578&repo=autoland&lineNumber=3050

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/68c50add73aa
Pre: Disable the startup cache when running a background task. r=dthayer
https://hg.mozilla.org/integration/autoland/rev/09db6a973d97
Pre: Disable extension manager in background tasks mode. r=mossop
https://hg.mozilla.org/integration/autoland/rev/b238efac91b4
Part 1: Wire background task mode into Gecko startup. r=mossop
https://hg.mozilla.org/integration/autoland/rev/14441979c730
Part 2: Add BackgroundTasksManager to invoke task defined in JS. r=mossop
https://hg.mozilla.org/integration/autoland/rev/73f56e24b72c
Part 3: Load a custom prefs file when running a background task. r=mossop,KrisWright
https://hg.mozilla.org/integration/autoland/rev/0e0e6bbf6f79
Post: Add test limiting the XPCOM graph of the no-op wait background task. r=mossop

Comment 22

[Sandor Molnar]

https://hg.mozilla.org/mozilla-central/rev/68c50add73aa
https://hg.mozilla.org/mozilla-central/rev/09db6a973d97
https://hg.mozilla.org/mozilla-central/rev/b238efac91b4
https://hg.mozilla.org/mozilla-central/rev/14441979c730
https://hg.mozilla.org/mozilla-central/rev/73f56e24b72c
https://hg.mozilla.org/mozilla-central/rev/0e0e6bbf6f79

Comment 23

[Nick Alexander :nalexander [he/him]]

Updating the ticket title to be more representative of what we've actually landed.

Comment 24

[Intermittent Failures Robot]

10 failures in 3891 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 10

Platform and build breakdown:

• linux64: 1
  • debug: 1
• macosx1014-64-shippable: 1
  • opt: 1
• macosx1014-64: 2
  • debug: 2
• windows7-32-shippable: 2
  • opt: 2
• windows10-64: 1
  • debug: 1
• windows7-32: 3
  • opt: 2
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1667276&startday=2021-01-25&endday=2021-01-31&tree=all