If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

objdir is being indexed in DXR

NEW
Unassigned

Status

Webtools
DXR
2 years ago
11 months ago

People

(Reporter: Dolske, Unassigned)

Tracking

Trunk

Firefox Tracking Flags

(firefox42 affected)

Details

(Reporter)

Description

2 years ago
I get double results for some queries, because DXR is indexing obj-x86_64-unknown-linux-gnu/* (this is the Firefox object dir, where build-time files are generated).

Eg: https://dxr.mozilla.org/mozilla-central/search?q=setDefaultBrowserDontAsk&redirect=false&case=false&limit=62&offset=0

Returns results for 4 files:


  browser/components/nsBrowserGlue.js
  browser/locales/en-US/chrome/browser/shellservice.properties
  obj-x86_64-unknown-linux-gnu/dist/bin/browser/chrome/en-US/locale/browser/shellservice.properties
  obj-x86_64-unknown-linux-gnu/dist/bin/browser/components/nsBrowserGlue.js

Only the first 2 results are useful.
See https://bugzilla.mozilla.org/show_bug.cgi?id=842547 for why we are indexing the objdir.

If these results are duplicated via symlink, then we will soon stop seeing the link versions thanks to https://bugzilla.mozilla.org/show_bug.cgi?id=1185687.

Updated

2 years ago
Duplicate of this bug: 1191682
(Reporter)

Comment 3

2 years ago
(In reply to Peter Elmers [:new_one] from comment #1)
> See https://bugzilla.mozilla.org/show_bug.cgi?id=842547 for why we are
> indexing the objdir.
> 
> If these results are duplicated via symlink, then we will soon stop seeing
> the link versions thanks to
> https://bugzilla.mozilla.org/show_bug.cgi?id=1185687.

What was MXR doing that made it work for the use-case in bug 842547? (The PPluginInstanceChild.cpp example doesn't work for me on MXR, so I'm not exactly sure what's being asked for.)

Skipping symlinks will help somewhat, but it still means that every pre-processed file will show up in objdir (for example, https://dxr.mozilla.org/mozilla-central/source/obj-x86_64-unknown-linux-gnu/dist/bin/browser/chrome/browser/content/browser/browser.js). Maybe that's useful for some generated platform stuff, for but front-end code I can't think of an example where that's ever useful (and it's usually going to be wrong/misleading, because the output is specific to Linux and whatever other #defines happen to be in play on DXR).

Maybe there's just a specific whitelist of files from objdir that Ehsan et al are interested in?
Flags: needinfo?(ehsan)

Comment 4

2 years ago
DXR gives you function call graphs for example, and the functions participating in such graphs can be in generated code, which is why we need to index those.  Those are limited to generated C++ headers and source files though, I don't think we should index everything in the objdir.
Flags: needinfo?(ehsan)
If a file has a unique name, we can type it in the search bar and press enter to open that file directly in DXR.

However, now that the objdir is indexed, most .idl and non-generated .h files are no longer unique and therefore can't be opened as above.

Indexing generated files in DXR is good, but I think it would be better to exclude those files that are already in the tree.

Comment 6

2 years ago
(In reply to Jonathan Hao [:jhao] from comment #5)
> If a file has a unique name, we can type it in the search bar and press
> enter to open that file directly in DXR.
> 
> However, now that the objdir is indexed, most .idl and non-generated .h
> files are no longer unique and therefore can't be opened as above.
> 
> Indexing generated files in DXR is good, but I think it would be better to
> exclude those files that are already in the tree.

Agreed.
Duplicate of this bug: 1193095
Comment hidden (metoo)

Comment 9

a year ago
I thought I would start to filter out objdir results mentally the more I used DXR, but on the contrary this is getting more and more annoying the more I use it.

It's still a minor annoyance, but I'd be glad to see it fixed. If fixing this in the back-end is difficult, maybe it can be worked around by filtering out paths starting with "obj-" on simple filename or full text queries?

How can I propose this bug for triage to the team working on DXR?
Flags: needinfo?(gps)
Ehsan, is there an easy way to get a list of the things that aren't "generated C++ headers and source files", as you cite above? DXR has an ignore option we could dump those into.
Flags: needinfo?(ehsan)
Also note that bug 1219399 may offer an eventual solution to those who want to easily filter out generated files. Feedback on that is welcome.
The build system could produce a machine-readable file of files relevant to DXR. DXR could then integrate with version control to find untracked files and ignore them unless they are in the build system's "relevant for indexing list."

If you'd like this feature, please file a Core :: Build Config bug and tell us which generated files are relevant to indexing (I have a good idea, but I'd prefer if a heavy DXR user actually weighed in).
Flags: needinfo?(gps)
Something like that sounds workable from the DXR perspective.

Comment 14

a year ago
Can't we simply consider any .h and .cpp file under your objdir that is not a symlink a generated source?

(We symlink a lot of things inside objdir/dist/include for example which we should probably ignore for this purpose.)
Flags: needinfo?(ehsan) → needinfo?(erik)
We could arrange for that. Can somebody familiar with the FF build system say whether that would suffice, or would we end up having to implement the more general bug 1219399 as well?
Flags: needinfo?(erik)
We may want to index more than just .h and .cpp files. Furthermore, I prefer the objdir be treated as a black box. This allows build system maintainers to minimize the theoretical surface area of the objdir "api" and refactor things as we see fit.

Let's have the build system output a machine readable file that DXR can read. It isn't too difficult to implement. And DXR will only need to know about the path to one file, not a list of file patterns or rules.

Someone who knows what they want indexed file a bug against Core :: Build config, assign to me, and I'll implement it.
Sounds good to me. I like not needing to have the DXR config continually change to accomodate the addition and removal of generated files in moz-central.
You need to log in before you can comment on or make changes to this bug.