AppConstants.sys.mjs is not indexed
Categories
(Webtools :: Searchfox, task)
Tracking
(Not tracked)
People
(Reporter: arai, Assigned: arai)
References
Details
Attachments
(3 files)
bug 1937699 added preprocessed files to the generated files list, and the preprocessed Readerable.sys.mjs is now indexed, but the preprocessed AppConstants.sys.mjs is not indexed for some reason.
Perhaps it's caused by the content being different between architecture?
Comment 1•26 days ago
|
||
(In reply to Tooru Fujisawa [:arai] from comment #0)
Perhaps it's caused by the content being different between architecture?
This is probably related, yeah:
[2025-01-20T17:05:54Z INFO tools::file_format::analysis] Error trying to open analysis file [analysis-*/__GENERATED__/./dist/bin/update-settings.ini]
Taking generated file ./dist/bin/modules/AppConstants.sys.mjs from linux64
Taking generated file ./dist/bin/modules/AppConstants.sys.mjs from macosx64
Taking generated file ./dist/bin/modules/AppConstants.sys.mjs from macosx64-aarch64
Taking generated file ./dist/bin/modules/AppConstants.sys.mjs from win64
Taking generated file ./dist/bin/modules/AppConstants.sys.mjs from android-aarch64
Taking generated file ./dist/bin/modules/AppConstants.sys.mjs from ios
Generated file ./dist/bin/modules/Readerable.sys.mjs was identical across platforms where it was created
And doing a find in m-c:
ubuntu@ip-172-31-28-166:~/index/mozilla-central$ find . -name AppConstants.sys.mjs\*
./git/toolkit/modules/AppConstants.sys.mjs
./description/toolkit/modules/AppConstants.sys.mjs
./analysis/toolkit/modules/AppConstants.sys.mjs.gz
./analysis/toolkit/modules/AppConstants.sys.mjs
./file/toolkit/modules/AppConstants.sys.mjs.gz
./file/toolkit/modules/AppConstants.sys.mjs
./detailed-per-file-info/toolkit/modules/AppConstants.sys.mjs
./objdir/__ios__/dist/bin/modules/AppConstants.sys.mjs
./objdir/__macosx64-aarch64__/dist/bin/modules/AppConstants.sys.mjs
./objdir/__android-aarch64__/dist/bin/modules/AppConstants.sys.mjs
./objdir/__linux64__/dist/bin/modules/AppConstants.sys.mjs
./objdir/__macosx64__/dist/bin/modules/AppConstants.sys.mjs
./objdir/__win64__/dist/bin/modules/AppConstants.sys.mjs
And then checking our files listings, this may not be lined up correctly:
ubuntu@ip-172-31-28-166:~/index/mozilla-central$ grep AppConstants.sys.mjs *.list
generated-files-android-aarch64.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files-ios.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files-linux64.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files-macosx64-aarch64.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files-macosx64.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files-win64.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files.list:./dist/bin/modules/AppConstants.sys.mjs
generated-files.list:./dist/bin/modules/AppConstants.sys.mjs
Assignee | ||
Comment 2•24 days ago
|
||
Looks like there's dependency between scripts that's not handled properly.
The objdir-dir
and objdir-files
are generated from the list of files in analysis
, which is created by find-objdir-files.sh
,
but at that point, the analysis for JS files aren't yet created, given they're generated by js-analyze.sh
, which is done after find-objdir-files.sh
.
So, the directory file/__GENERATED__/__linux64__/dist/bin
is not created by objdir-mkdirs.sh
.
I guess, we can reorder some of them.
$MOZSEARCH_PATH/scripts/find-objdir-files.sh
$MOZSEARCH_PATH/scripts/objdir-mkdirs.sh
date
URL_MAP_PATH=$INDEX_ROOT/aliases/url-map.json
DOC_TREES_MAP=$INDEX_ROOT/doc-trees.json
$MOZSEARCH_PATH/scripts/process-chrome-map.py $GIT_ROOT $URL_MAP_PATH $INDEX_ROOT/*.chrome-map.json\
|| handle_tree_error "process-chrome-map.py"
date
$MOZSEARCH_PATH/scripts/js-analyze.sh $CONFIG_FILE $TREE_NAME || handle_tree_error "js-analyze.sh"
Assignee | ||
Comment 3•24 days ago
|
||
The script that depends on objdir-*
seems to be only ipdl-analyze.sh
.
So maybe we can move the find-objdir-files.sh
+objdir-mkdirs.sh
immediately before that, or maybe create a temporary list and then re-run those scripts to generate the final list.
I'll see how it goes locally.
Assignee | ||
Comment 4•24 days ago
|
||
Also realized that the generated files are not processed by other analysis, such as js-analyze.sh
.
That part also needs to be solved if we rely on the existence of analysis files.
The other option would be to directly reflect the files in objdir
, instead of using analysis files.
Comment 5•24 days ago
|
||
(In reply to Tooru Fujisawa [:arai] from comment #4)
The other option would be to directly reflect the files in
objdir
, instead of using analysis files.
If this simplifies things, this seems good. We do already prioritize the file itself instead of the analysis file in the merge process, so improving consistency around that makes sense, especially if it's causing these problems.
Assignee | ||
Comment 6•23 days ago
|
||
Using objdir file list looks working.
At least there's no missing files/dirs in the list.
There are some newly added files/dirs (see the attachment).
I'll test some more (and fix the test repo around the expecation or the intermediate files) and post PR.
Comment 7•23 days ago
|
||
Neat, those all seem helpful. The __RUST_BUILD_SCRIPT__
paths in particular are probably quite nice to have!
The only thing that seems like it might not be desired is __GENERATED__/rust-analyzed
but that's correctly there because we intentionally put it there but we no longer should be putting it there because the change to use rust-analyzer and SCIP and that we don't automatically run rust-analyzer means that we don't need a marker file to tell us not to automatically run it. That is, we should remove those lines that touch the file.
Assignee | ||
Comment 8•22 days ago
|
||
The patch to use the actual files in objdir is now available on https://arai.searchfox.org/
The preprocessed AppConstants.sys.mjs files are shown in the search, but not yet analyzed. I'll handle it in separate bug.
https://arai.searchfox.org/mozilla-central/search?q=&path=AppConstants.sys.mjs&case=false®exp=false
The rust bindings are also shown (also not analyzed yet)
https://arai.searchfox.org/mozilla-central/source/__GENERATED__/__linux64__/__RUST_BUILD_SCRIPT__/gecko-profiler/gecko/bindings.rs
Assignee | ||
Comment 9•22 days ago
|
||
Assignee | ||
Comment 10•22 days ago
|
||
for the rust-analyzed file part in the mozsearch-mozilla repo
Assignee | ||
Updated•19 days ago
|
Description
•