1635255 - IPC Implementation for Project FOG

Status: RESOLVED FIXED

(Toolkit :: Telemetry, task, P1)

Description

[Chris H-C :chutten]

Implement the design from bug 1618253.

• Build and document and test the generic mechanism for passing information across processes at the scheduled moments
• Augment at least one metric type with IPC capabilities to prove out the usefulness of the documentation and the comprehensiveness of the testing
• Establish best practices (a How To Guide?) about how to think about IPC when it comes to metric types in FOG (including where and how to best document that certain operations may not work out so well due to them not being commutative operations).

Anti-goals:

• Don't implement all of the rust metric types across IPC just yet. We don't need them all until Milestone 6 after all. But you can file a bunch of bugs for them if you'd like.

Comment 1

[Chris H-C :chutten]

Had a long and fruitful chat in https://chat.mozilla.org/#/room/#ipc:mozilla.org about IPC and how best to accomplish what we're after. Unfortunately, the present state of things forbids us from being perfect perf-respecting citizens in IPC land.

So for now it's okay to implement the few things we need on PContent (on the main thread). But that means we need to implement with an eye towards replacing this state of things with a better solution when it should become available.

I've added a task to Milestone 8 in the roadmap to re-evaluate the state of things in October. I've also filed bug 1641989. Until then, let's move on with implementation with the following things in mind:

• The IPC backend will be replaced, so architect the code around that assumption
• Comments in the code and Documentation in the tree should reference this comment and bug 1641989 and be clear that this is a temporary state
• We should do our best to be good citizens while we're on PContent: keep message sizes low. Dispatch as quickly as possible.

Comment 2

[Chris H-C :chutten]

Attachment: Bug 1635255 - Add FOG IPC to PContent r?janerik!

Temporarily on PContent instead of managed by PBackground, there's one
parentbound message for occasionally uplifting Glean data from child processes
and one childbound message for forcing the immediate flush of Glean data in the
async return.

Can't write gtests for this as ContentChild and ContentParent include things
that aren't present in gtest.

Comment 3

[Chris H-C :chutten]

Attachment: Bug 1635255 - Sketch out StringMetric ipc support r?janerik!

Depends on D78077

Comment 4

[Chris H-C :chutten]

Attachment: Bug 1635255 - Sketch out StringMetric ipc support r?janerik!

This isn't 100% of a solution yet as the main process has no way of turning
a MetricId into a StringMetric (we're gonna a need a table. Maybe several.)
which we'll need before we can replay.

Also, there's no serialization or communication yet, which I guess is a bigger
deal.

Oh, and no detection of which process we're even on.

Depends on D78077

Comment 5

[Chris H-C :chutten]

Attachment: Bug 1635255 - Flesh out FOG IPC one additional layer r?janerik!

Introducting FOGIPC, the central clearinghouse for the C++ layer between
PContent and FOG's Rust impl.

Gotta add tests.

Depends on D79741

Comment 6

[Chris H-C :chutten]

Attachment: Bug 1635255 - Add skeleton FOG IPC GTests r?janerik!

Depends on D79742

Comment 7

[Chris H-C :chutten]

Attachment: Bug 1635255 - Build glean crate conditionally on Gecko symbols r?janerik!

Introduces a gkrust Cargo feature glean_with_gecko and with_gecko on fog
and glean. This feature signifies the presence of gecko symbols in the build.

Use this feature to implement needs_ipc() which asks Gecko which process type
we're running as.

Depends on D79743

Comment 8

[Chris H-C :chutten]

Attachment: Bug 1635255 - Document FOG's IPC r?janerik!

Depends on D79744

Comment 9

[Chris H-C :chutten]

Attachment: Bug 1635255 - Skeleton IPC impl for Counter r?janerik!

Depends on D79745

Comment 10

[Chris H-C :chutten]

Attachment: Bug 1635255 - Implement the Rust <-> C++ ffi for FOG IPC r?janerik!

Bringing it all together.

Depends on D79746

Comment 11

[Chris H-C :chutten]

Attachment: Bug 1635255 - Test child-process counter metrics r?janerik!

Depends on D79747

Comment 12

[Chris H-C :chutten]

Attachment: Bug 1635255 - Adapt to get_AppInfoService removal r?janerik!

Depends on D79898

Comment 13

[Chris H-C :chutten]

Attachment: Bug 1635255 - Don't compile FOG IPC layer and tests if not MOZ_GLEAN r?janerik!

Depends on D80516

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0862a33399cf
Add FOG IPC to PContent r=janerik
https://hg.mozilla.org/integration/autoland/rev/49447f10ad6e
Sketch out StringMetric ipc support r=janerik
https://hg.mozilla.org/integration/autoland/rev/bfcfda9cb19d
Flesh out FOG IPC one additional layer r=janerik
https://hg.mozilla.org/integration/autoland/rev/5fc5e1070d4d
Add skeleton FOG IPC GTests r=janerik
https://hg.mozilla.org/integration/autoland/rev/d0591dc8d5a1
Build glean crate conditionally on Gecko symbols r=janerik
https://hg.mozilla.org/integration/autoland/rev/cb54f7a3177d
Document FOG's IPC r=janerik
https://hg.mozilla.org/integration/autoland/rev/9b79c8208144
Skeleton IPC impl for Counter r=janerik
https://hg.mozilla.org/integration/autoland/rev/4d0c4beb910e
Implement the Rust <-> C++ ffi for FOG IPC r=janerik
https://hg.mozilla.org/integration/autoland/rev/4675772344eb
Test child-process counter metrics r=janerik
https://hg.mozilla.org/integration/autoland/rev/27df18486bff
Adapt to get_AppInfoService removal r=janerik
https://hg.mozilla.org/integration/autoland/rev/d3e93edb1c76
Don't compile FOG IPC layer and tests if not MOZ_GLEAN r=janerik

Comment 15

[Alexandru Michis [:malexandru]]

Backed out 11 changesets (Bug 1635255) for causing bustages in FOGIPC.cpp

Backout link: https://hg.mozilla.org/integration/autoland/rev/5794fe9ac1e5baf6a4d537803c5bbc7b8bdd5277

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception%2Crunnable&tochange=5794fe9ac1e5baf6a4d537803c5bbc7b8bdd5277&fromchange=f4b0a007b4ff3a43fecdfa31bff2a966352d79c9&selectedTaskRun=OwMTNiEgRZ6AA-rTBi93xA.0

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=307384546&repo=autoland&lineNumber=43253

Comment 16

[Chris H-C :chutten]

Well that's odd, I pushed this to try yesterday and it built just fine: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=e5dfb4389a0034812d76162ce6425df44f6b815f

Lemme rebase and try building again, I guess?

Comment 17

[Chris H-C :chutten]

Ah-ha! Culprit is bug 1637500 renaming GetCurrentThreadSerialEventTarget to GetCurrentSerialEventTarget landing juuuuusssst before mine did. (Well, some hours before. But still. Dang close.) New patches a-comin'.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2dd4b408b19d
Add FOG IPC to PContent r=janerik
https://hg.mozilla.org/integration/autoland/rev/d39f6f55fed1
Sketch out StringMetric ipc support r=janerik
https://hg.mozilla.org/integration/autoland/rev/395c9a05d024
Flesh out FOG IPC one additional layer r=janerik
https://hg.mozilla.org/integration/autoland/rev/95fd1634efbd
Add skeleton FOG IPC GTests r=janerik
https://hg.mozilla.org/integration/autoland/rev/ad489260453a
Build glean crate conditionally on Gecko symbols r=janerik
https://hg.mozilla.org/integration/autoland/rev/b431e8c9adf0
Document FOG's IPC r=janerik
https://hg.mozilla.org/integration/autoland/rev/ae8543c87d15
Skeleton IPC impl for Counter r=janerik
https://hg.mozilla.org/integration/autoland/rev/704a72c948ba
Implement the Rust <-> C++ ffi for FOG IPC r=janerik
https://hg.mozilla.org/integration/autoland/rev/122fa4a0e054
Test child-process counter metrics r=janerik
https://hg.mozilla.org/integration/autoland/rev/5690707891fe
Adapt to get_AppInfoService removal r=janerik
https://hg.mozilla.org/integration/autoland/rev/057f6d9463e8
Don't compile FOG IPC layer and tests if not MOZ_GLEAN r=janerik

Comment 19

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/2dd4b408b19d
https://hg.mozilla.org/mozilla-central/rev/d39f6f55fed1
https://hg.mozilla.org/mozilla-central/rev/395c9a05d024
https://hg.mozilla.org/mozilla-central/rev/95fd1634efbd
https://hg.mozilla.org/mozilla-central/rev/ad489260453a
https://hg.mozilla.org/mozilla-central/rev/b431e8c9adf0
https://hg.mozilla.org/mozilla-central/rev/ae8543c87d15
https://hg.mozilla.org/mozilla-central/rev/704a72c948ba
https://hg.mozilla.org/mozilla-central/rev/122fa4a0e054
https://hg.mozilla.org/mozilla-central/rev/5690707891fe
https://hg.mozilla.org/mozilla-central/rev/057f6d9463e8