1456845 - IPDL analysis is broken

Status: RESOLVED FIXED

(Webtools :: Searchfox, defect)

Description

[Kartikaya Gupta (email:[e-mail])]

Going to an IPDL file like https://searchfox.org/mozilla-central/source/gfx/layers/ipc/PLayerTransaction.ipdl and clicking on function names should produce links to find callers and such, but it doesn't. Looking at the indexer log I see this:

+ xargs /home/ubuntu/mozsearch/tools/target/release/ipdl-analyze -I accessible/ipc/other -I devtools/shared/heapsnapshot -I dom/asmjscache -I dom/broadcastchannel -I dom/cache -I dom/clients/manager -I dom/fetch -I dom/file/ipc -I dom/filehandle -I dom/filesystem -I dom/gamepad/ipc -I dom/indexedDB -I dom/ipc -I dom/media/gmp -I dom/media/ipc -I dom/media/systemservices -I dom/media/webrtc -I dom/media/webspeech/synth/ipc -I dom/messagechannel -I dom/midi -I dom/network -I dom/payments/ipc -I dom/plugins/ipc -I dom/presentation/ipc -I dom/quota -I dom/serviceworkers -I dom/storage -I dom/webauthn -I dom/webbrowserpersist -I extensions/spellcheck/hunspell/glue -I gfx/ipc -I gfx/layers/ipc -I gfx/vr/ipc -I hal/sandbox -I ipc/glue -I ipc/testshell -I js/ipc -I js/xpconnect/loader -I layout/base -I layout/ipc -I layout/printing/ipc -I media/mtransport/ipc -I netwerk/cookie -I netwerk/dns -I netwerk/ipc -I netwerk/protocol/ftp -I netwerk/protocol/http -I netwerk/protocol/websocket -I netwerk/protocol/wyciwyg -I security/manager/ssl -I toolkit/components/backgroundhangmonitor -I toolkit/components/extensions/webrequest -I toolkit/components/printingui/ipc -I tools/profiler/gecko -I uriloader/exthandler -I uriloader/prefetch -I widget -I widget/gtk -I widget/headless -d /index/mozilla-central/analysis/__GENERATED__/ipc/ipdl/_ipdlheaders -b /index/mozilla-central/gecko-dev -a /index/mozilla-central/analysis
Parsing file /index/mozilla-central/gecko-dev/accessible/ipc/other/PDocAccessible.ipdl
Parsing file /index/mozilla-central/gecko-dev/devtools/shared/heapsnapshot/PHeapSnapshotTempFileHelper.ipdl
Parsing file /index/mozilla-central/gecko-dev/dom/asmjscache/PAsmJSCacheEntry.ipdl
Parsing file /index/mozilla-central/gecko-dev/dom/broadcastchannel/PBroadcastChannel.ipdl
Parsing file /index/mozilla-central/gecko-dev/dom/cache/CacheTypes.ipdlh
Parsing file /index/mozilla-central/gecko-dev/dom/cache/PCache.ipdl
Parsing file /index/mozilla-central/gecko-dev/dom/cache/PCacheOp.ipdl
Parsing file /index/mozilla-central/gecko-dev/dom/cache/PCacheStorage.ipdl
Parsing file /index/mozilla-central/gecko-dev/dom/cache/PCacheStreamControl.ipdl
/index/mozilla-central/gecko-dev/dom/cache/PCacheStreamControl.ipdl :/index/mozilla-central/gecko-dev/dom/cache/PCacheStreamControl.ipdl 11:17 Error: Unrecognized token `class'.
Specification could not be parsed.


i.e. the line at https://searchfox.org/mozilla-central/rev/36dec78aecc40539ecc8d78e91612e38810f963c/dom/cache/PCacheStreamControl.ipdl#11 is causing the analyzer to die.

Although... I would still expect the successfully parsed files (e.g. PCache.ipdl) to get linked properly and that doesn't seem to be happening either.

Comment 1

[Andrew McCreight [:mccr8]]

I can fix the parser.

Comment 2

[Kartikaya Gupta (email:[e-mail])]

Thanks!

Comment 3

[Andrew McCreight [:mccr8]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #0)
> Although... I would still expect the successfully parsed files (e.g.
> PCache.ipdl) to get linked properly and that doesn't seem to be happening
> either.

IIRC, the way I set up the Rust IPDL parser, it bails immediately when a file fails to parse. That makes sense in the context of a Firefox build, but not in the sense of Searchfox. IPDL parsing only breaks once or twice a year so I haven't devoted any real thought to figuring out precisely how to fix that, or if we even should.

Comment 4

[Kartikaya Gupta (email:[e-mail])]

Ah, I see. I'm fine with it as-is, I was just confused because it didn't match what I was expecting.

Comment 5

[Andrew McCreight [:mccr8]]

I updated my upstream Rust IPDL parser repo with fixes: https://github.com/amccreight/ipdl_parser

In addition to bug 1443954, I think bug 1379680 broke the IPDL parser, so it is possible it hasn't been working for a number of months. Anyways, I confirmed that the Rust IPDL parser can once again parse all Firefox IPDL files. I'll dig up my SearchFox repo and do a pull request with a subrepo update.

Comment 6

[Andrew McCreight [:mccr8]]

https://github.com/mozsearch/mozsearch/pull/107

Comment 7

[Kartikaya Gupta (email:[e-mail])]

This is deployed now, thanks!