1487583 - Add analysis for macOS code using taskcluster job

Status: RESOLVED FIXED

(Webtools :: Searchfox, enhancement)

Description

[Kartikaya Gupta (email:[e-mail])]

After bug 1425597 is done it's pretty easy to also download the macOS analysis data from the macOS taskcluster job and combine it into the linux stuff. That way C++ and rust code behind macOS compile guards will get indexed too. I'm pretty close to having this done (just hammering out last couple of tweaks now).

The only open question is what to do about generated sources where the generated file looks different on different platforms. There's at least one of these. For now I'm just going to use the linux variant of the file but maybe we can expand the __GENERATED__ folder to be like __GENERATED__/__LINUX__/ and __GENERATED__/__MACOSX__/ instead. Although the majority of the generated files are the same and so maybe this is overkill?

Comment 1

[Kartikaya Gupta (email:[e-mail])]

https://github.com/staktrace/mozsearch-mozilla/commits/tc has the current WIP

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

What file looks different for the different platforms / how different does it look?

Comment 3

[Kartikaya Gupta (email:[e-mail])]

(In reply to Kartikaya Gupta (email:[e-mail]) (parental leave) from comment #1)
> https://github.com/staktrace/mozsearch-mozilla/commits/tc has the current WIP

https://github.com/mozsearch/mozsearch/pull/137 is also needed. Current instance on dev.searchfox.org has the macOS indexing stuff.

(In reply to Andrew Sutherland [:asuth] from comment #2)
> What file looks different for the different platforms / how different does
> it look?

I downloaded the target.generated-files.tar.gz files from the two idx jobs at [1] and diffed the contents. These files exist in both but with different contents:

gfx/cairo/cairo/src/cairo-features.h
ipc/ipdl/IPCMessageTypeName.cpp
ipc/ipdl/PCompositorWidget.cpp
ipc/ipdl/PCompositorWidgetChild.cpp
ipc/ipdl/PCompositorWidgetParent.cpp
ipc/ipdl/PlatformWidgetTypes.cpp
ipc/ipdl/_ipdlheaders/mozilla/layers/PCompositorBridgeChild.h
ipc/ipdl/_ipdlheaders/mozilla/layers/PCompositorBridgeParent.h
ipc/ipdl/_ipdlheaders/mozilla/widget/PCompositorWidget.h
ipc/ipdl/_ipdlheaders/mozilla/widget/PCompositorWidgetChild.h
ipc/ipdl/_ipdlheaders/mozilla/widget/PCompositorWidgetParent.h
ipc/ipdl/_ipdlheaders/mozilla/widget/PlatformWidgetTypes.h
mozilla-config.h
toolkit/components/telemetry/TelemetryHistogramEnums.h
xpcom/reflect/xptinfo/xptdata.cpp
xpcom/xpcom-config.h

The differences are certainly not trivial - thousands of lines are different, with large hunks added/removed.

[1] https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=09de350001f07965f787c19f298fd0cd6cd839fa&filter-searchStr=searchfox

Comment 4

[Kartikaya Gupta (email:[e-mail])]

https://github.com/mozsearch/mozsearch-mozilla/pull/12

Comment 5

[Kartikaya Gupta (email:[e-mail])]

This is deployed now.

Comment 6

[(no longer active)]

Firstly, fantastic work, thanks so much!

> It shouldn't be hard to add the macOS version as well, but I'm not sure
> what a good UX would be for exposing that. If you have suggestions please
> leave them in bug 1487583.

I wonder, would it make sense to allow folks to choose a default platform and persist that choice in a cookie and select which generated code to show/search in based on that?  For example, by default use Linux64, but if someone develops on Windows heavily they would be able to set up Searchfox to show them Windows code by default.

Comment 7

[Kartikaya Gupta (email:[e-mail])]

(In reply to :Ehsan Akhgari from comment #6)
> I wonder, would it make sense to allow folks to choose a default platform
> and persist that choice in a cookie and select which generated code to
> show/search in based on that?  For example, by default use Linux64, but if
> someone develops on Windows heavily they would be able to set up Searchfox
> to show them Windows code by default.

That sounds like a reasonable solution, yeah. I've filed bug 1490853 for it.

Comment 8

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

It's interesting that there are large diffs in the IPDL generated files.  Generally we've tried to avoid platform-specific IPDL, preferring stub implementations instead, but there are some exceptions (e.g., accessible/ipc/win vs. accessible/ipc/other, and it looks like widget/ does something similar), and of course searchfox has to deal with the code as it exists, not as we'd like it to be.