418089 - MXR doesn't recognize preprocessor comments ("# ") as such and indexes license headers

Status: RESOLVED FIXED

(Webtools Graveyard :: MXR, defect)

Description

[Steffen Wilberg]

License headers are quite often formatted as preprocessor comments, i.e. each line starts with "# ". Contrary to /* */ comments, MXR doesn't recognize these as comments and indexes the whole license header.

Example: http://mxr.mozilla.org/seamonkey/source/toolkit/mozapps/downloads/content/downloads.js

Comment 1

[timeless]

yes, it doesn't, but they aren't valid js, and # does mean something in js. I haven't considered this important enough to fix. Sure there are some extra shiny words, but what harm do they cause?

Comment 2

[Steffen Wilberg]

Are you sure "# " at the line beginning is valid ja?

This
- clutters indent results, e.g. http://mxr.mozilla.org/seamonkey/ident?i=contents and http://mxr.mozilla.org/seamonkey/ident?i=replace
- increases the size of the html pages
- might affect performance of mxr

Comment 3

[timeless]

changeset 102	e25247bf901d

Comment 4

[Steffen Wilberg]

That's http://hg.mozilla.org/webtools/index.cgi/mxr/rev/e25247bf901d, which excludes the "BEGIN LICENSE BLOCK"-"END LICENSE BLOCK" block.

But that doesn't fix preprocessor comments outside the license block like http://mxr.mozilla.org/mozilla-central/source/toolkit/components/help/content/help.js#100.

Comment 5

[timeless]

So, this was a compromise, and it's partially because I don't want to teach MXR too many Mozilla specific hacks. MXR is also used to index OpenSolaris, Maemo, MeeGo, and Symbian, each of which is rather large...

The explanation goes something like this:
http://mxr.mozilla.org/mozilla-central/ident?i=getXulWin&tree=mozilla-central&filter=help.js
line 783 -- function getXulWin()
line 782 -- # getXulWin - Returns the current Help window as a nsIXULWindow.
line 805 -- var xulwin = getXulWin();

http://mxr.mozilla.org/mozilla-central/ident?i=nsIXULWindow&filter=help.js
line 782 -- # getXulWin - Returns the current Help window as a nsIXULWindow.
line 791 -- return ifreq.getInterface(Components.interfaces.nsIXULWindow);

http://mxr.mozilla.org/mozilla-central/ident?i=setHelpFileURI&strict=1
line 40 -- # setHelpFileURI function to set this value.
line 80 -- function setHelpFileURI(rdfURI)
line 79 -- # setHelpFileURI - Sets the default content pack to use in the Help Viewer

If an identifier is used in a comment, it's possible that the comment is actually explaining what the identifier does or how it's used. Yes it isn't actually a use, but it can be useful information (or a flag to some out of date comment which should be fixed...)

http://mxr.mozilla.org/mozilla-central/ident?i=Help&strict=1

line 76 -- printf("Template: Help(): display your help message(s) here");

If a token is only used in comments, then it won't be recognized as an identifier and won't be indexed.

This means that "current" has a 'bad' hit:
http://mxr.mozilla.org/mozilla-central/ident?i=current&filter=help.js
But "reduction" doesn't:
http://mxr.mozilla.org/mozilla-central/ident?i=reduction&filter=help.js

I'm going to leave this bug as resolved fixed. If you really want to file a bug asking for general comments to be ignored, you're welcome to do so (however, please don't expect me to fix it). This bug really was fixed according to your original report in comment 0.

Comment 6

[Steffen Wilberg]

But why do yo consider "# " style comments to be of more value than // and /* */ style comments?

Lines starting with "# " are being stripped by the preprocessor, so they're not visible in view-source when clicking on a link in the error console, yet mxr indexes them.

// and /* */ style comments remain in the shipped product, yet mxr doesn't index them.

And in e.g. browser.js (http://mxr.mozilla.org/mozilla-central/source/browser/base/content/browser.js), the license header is still indexed despite having
# ***** BEGIN LICENSE BLOCK ***** in line 2.

Comment 7

[timeless]

Ah, that's more of a bug the other way around. Someday when mxr's indexer is faster, I'll fix it so that normal comments don't suppress identifier use.