Also: in terms of the clang-plugin, one spot that I thought would use up big gobs of memory is [this code](https://searchfox.org/mozilla-central/rev/9193635dca8cfdcb68f114306194ffc860456044/build/clang-plugin/mozsearch-plugin/MozsearchIndexer.cpp#534-536) where we read the entire analysis file for a source file into memory, append the new rows, and sort it all before writing it back out. If the analysis file for a source file gets really big this would eat up a lot of memory. I looked at a recent index of m-c to find the largest files in the win64 mozsearch-index.zip file, and got this: ``` $ find . -type f -print0 | xargs -0 wc -c | sort -rn | head 125440912 ./__GENERATED__/dist/include/mozilla/Assertions.h 67277607 ./js/src/zydis/Zydis/Generated/InstructionDefinitions.inc 58214307 ./__GENERATED__/dom/bindings/TestJSImplGenBinding.cpp 43461257 ./__GENERATED__/dom/bindings/TestCodeGenBinding.cpp 36553630 ./third_party/sqlite3/src/sqlite3.c 35245726 ./__GENERATED__/dom/bindings/CSS2PropertiesBinding.cpp 32859531 ./__GENERATED__/dist/include/nsISupportsImpl.h 26875546 ./__GENERATED__/dom/bindings/TestExampleGenBinding.cpp 22401566 ./__GENERATED__/dist/include/mozilla/Likely.h 19753412 ./__GENERATED__/toolkit/components/telemetry/TelemetryHistogramData.inc 19729881 ./__GENERATED__/dist/include/mozilla/StaticAnalysisFunctions.h 19566712 ./toolkit/components/reputationservice/chromium/chrome/common/safe_browsing/csd.pb.h ``` So Assertions.h generates an analysis file 125M big. I don't know how that compares to what else clang is doing and how much memory is available on the system.
Bug 1638664 Comment 5 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Also: in terms of the clang-plugin, one spot that I thought would use up big gobs of memory is [this code](https://searchfox.org/mozilla-central/rev/9193635dca8cfdcb68f114306194ffc860456044/build/clang-plugin/mozsearch-plugin/MozsearchIndexer.cpp#534-536) where we read the entire analysis file for a source file into memory, append the new rows, and sort it all before writing it back out. If the analysis file for a source file gets really big this would eat up a lot of memory. I looked at a recent index of m-c to find the largest files in the win64 mozsearch-index.zip file, and got this: ``` $ find . -type f -print0 | xargs -0 wc -c | sort -rn | head # approximately what I did 125440912 ./__GENERATED__/dist/include/mozilla/Assertions.h 67277607 ./js/src/zydis/Zydis/Generated/InstructionDefinitions.inc 58214307 ./__GENERATED__/dom/bindings/TestJSImplGenBinding.cpp 43461257 ./__GENERATED__/dom/bindings/TestCodeGenBinding.cpp 36553630 ./third_party/sqlite3/src/sqlite3.c 35245726 ./__GENERATED__/dom/bindings/CSS2PropertiesBinding.cpp 32859531 ./__GENERATED__/dist/include/nsISupportsImpl.h 26875546 ./__GENERATED__/dom/bindings/TestExampleGenBinding.cpp 22401566 ./__GENERATED__/dist/include/mozilla/Likely.h 19753412 ./__GENERATED__/toolkit/components/telemetry/TelemetryHistogramData.inc 19729881 ./__GENERATED__/dist/include/mozilla/StaticAnalysisFunctions.h 19566712 ./toolkit/components/reputationservice/chromium/chrome/common/safe_browsing/csd.pb.h ``` So Assertions.h generates an analysis file 125M big. I don't know how that compares to what else clang is doing and how much memory is available on the system.