1719560 - Collect last_modified values for all RemoteSettings dumps in last_modified.json

Status: RESOLVED FIXED

(Firefox :: Remote Settings Client, task)

Description

[Rob Wu [:robwu]]

Currently, there is no central place where a RemoteSettings dump can be registered.
A dump is only packaged with Firefox if all of the following conditions are true:

• A JSON file is added to the tree.
• The JSON file is registered in moz.build to FINAL_TARGET_FILES.defaults.settings (also used outside m-c, by Thunderbird)
• The defaults/settings/<bucket>/<collection>.json file is matched by a pattern in the package-manifest.in file for the build target.
• (the JSON file in the tree may be updated automatically twice a week by periodic_file_updates.sh when the bucket+collection is returned by the RemoteSettings server)

In bug 1717068 it became apparent that there is a need for a central registry that summarizes metadata for all JSON dumps (specifically the last_modified timestamps of all dumps). To get the prototype working, I hardcoded the specific use case in a build script, but to support the general case (bug 1718083) and reduce the maintenance burden, the build script should use the same set of files that are registered in moz.build.

The build script should also account for the target-specific conditionals from package-manifest.in. This can be done by either moving the target-specific conditions to moz.build, or by somehow applying the rules from package-manifest.in to the list of files from moz.build.

Comment 1

[Graham Beckley [:grahamalama]]

Attachment: Bug 1719560 - Add password-recipes to last_modified.json r?leplatrem

Comment 2

[Rob Wu [:robwu]]

This bug is supposed to be used for addressing the issue in a generic way. Please file a new bug (feel free to link it to this bug) if you want to include specific dumps. And unit tests would be nice, to verify that the behavior works as desired.

Comment 3

[Graham Beckley [:grahamalama]]

Hey Rob. Sorry for this noise. -- [:leplatrem] and I were using this bug as an example of how to contribute to Firefox, so we only pushed a draft version of a patch for this bug to get me used to the development workflow. I do intend to flesh this out to make the solution generic and add a test or tests as necessary.

Comment 4

[Mathieu Leplatre [:leplatrem]]

Attachment: Bug 1719560 - Collect timestamps for all packaged dumps r?robwu

Comment 5

[Mathieu Leplatre [:leplatrem]]

Attachment: Bug 1719560 - Collect timestamps for all collections r?robwu

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6e11829eec98
Collect timestamps for all collections r=robwu

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/63407a7e0e90
Collect timestamps for all collections r=robwu

Comment 8

[Noemi Erli[:noemi_erli]]

Backed out 2 changesets (Bug 1719560, Bug 1725660) for causing build bustages CLOSED TREE

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&selectedTaskRun=GdQtekJWS_K4tC-xj8g-aA.0&resultStatus=testfailed%2Cbusted%2Cexception&searchStr=windows%2C2012%2Cx64%2Cdebug%2Cbuild-win64-plain%2Fdebug%2Cbp&revision=63407a7e0e90de8e1fade0933a181441477c9ff8

Log: https://treeherder.mozilla.org/logviewer?job_id=376066410&repo=autoland&lineNumber=90908

Backout: https://hg.mozilla.org/integration/autoland/rev/055c8d5e8d98f6b62e00cf5a40950bbc137e0c55

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0203682dddbe
Collect timestamps for all collections r=robwu

Comment 10

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/0203682dddbe