xpidllex.py and xpidlyacc.py are missing in Gecko10.0 for both Win & Mac

RESOLVED FIXED in mozilla13

Status

()

Core
Build Config
RESOLVED FIXED
5 years ago
2 years ago

People

(Reporter: paddy, Assigned: glandium)

Tracking

Trunk
mozilla13
x86
Windows 7
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [qa-][read Comment 17 if you're hitting this])

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
User Agent: Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11

Steps to reproduce:

I am trying to generate header and type lib using Gecko10.0(http://ftp.mozilla.org/pub/mozilla.org/xulrunner/releases/10.0/sdk/xulrunner-10.0.en-US.win32.sdk.zip) for my XPCOM to support FF10.

I am trying following commands:
...\xulrunner-sdk\sdk\bin>python header.py ISample.idl -I..\..\idl -o ISample.h
...\xulrunner-sdk\sdk\bin>python typelib.py ISample.idl -I..\..\idl -o ISample.xpt


Actual results:

I am getting following errors:

...\xulrunner-sdk\sdk\bin>python header.py ISample.idl -I..\..\idl -o ISample.h
Traceback (most recent call last):
  File "header.py", line 497, in <module>
    p = xpidl.IDLParser(outputdir=options.cachedir)
  File "...\xulrunner-sdk\sdk\bin\xpidl.py", line 1453, in __init__
    optimize=1)
  File "...\xulrunner-sdk\sdk\bin\ply\lex.py", line 1004, in lex
    lexobj.writetab(lextab,outputdir)
  File "...\xulrunner-sdk\sdk\bin\ply\lex.py", line 175, in writetab
    filename = os.path.join(outputdir,basetabfilename)+".py"
  File "C:\Python25\lib\ntpath.py", line 90, in join
    assert len(path) > 0
TypeError: object of type 'NoneType' has no len()

#################################################################################

...\xulrunner-sdk\sdk\bin>python typelib.py ISample.idl -I..\..\idl -o ISample.xpt
Traceback (most recent call last):
  File "typelib.py", line 313, in <module>
    p = xpidl.IDLParser(outputdir=options.cachedir)
  File "...\xulrunner-sdk\sdk\bin\xpidl.py", line 1453, in __init__
    optimize=1)
  File "...\xulrunner-sdk\sdk\bin\ply\lex.py", line 1004, in lex
    lexobj.writetab(lextab,outputdir)
  File "...\xulrunner-sdk\sdk\bin\ply\lex.py", line 175, in writetab
    filename = os.path.join(outputdir,basetabfilename)+".py"
  File "C:\Python25\lib\ntpath.py", line 90, in join
    assert len(path) > 0
TypeError: object of type 'NoneType' has no len()


Expected results:

On analysis I have found the xpidllex.py and xpidlyacc.py are missing in Gecko10.0. I have copied then from Gecko9.0 to Gecko10.0 and tried to generate the header and type and they worked fine. Then I built my XPCOM with these header and typelib and XPCOM now works fine with FF10.

So following are my queries:

Why xpidllex.py and xpidlyacc.py are missing in Gecko10.0?
Is there any other way to generate header and typelib for Gecko10.0?
Is use of xpidllex.py and xpidlyacc.py from Gecko9.0 in Gecko10.0 harmful?

Please suggest.
As I understand it, bug 691781 removed the checked in copies, they're now generated as part of the build.
Using the xpidllex.py and xpidlyacc.py from Gecko 9 will work fine as a workaround.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Wasn't there another bug on this (or was that a mailing list post)?
Yeah, I think so, but I can't find it.
Bug 722632, I don't know which way to dup this.
Duplicate of this bug: 722632
(Reporter)

Comment 7

5 years ago
Please let me know resolution on this issue. 

xpidllex.py and xpidlyacc.py are missing in Gecko11.0 too.
Assignee: nobody → khuey
Status: NEW → ASSIGNED
Duplicate of this bug: 714138
(Assignee)

Comment 9

5 years ago
Created attachment 598235 [details] [diff] [review]
Ship xpidllex.py and xpidlyacc.py
Attachment #598235 - Flags: review?(ted.mielczarek)
Attachment #598235 - Flags: review?(ted.mielczarek) → review+
(Assignee)

Comment 10

5 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/013b99c7ae65
Assignee: khuey → mh+mozilla
(Assignee)

Updated

5 years ago
status1.9.2: --- → unaffected
status-firefox-esr10: --- → affected
status-firefox10: --- → affected
status-firefox11: --- → affected
status-firefox12: --- → affected
status-firefox13: --- → fixed
Target Milestone: --- → mozilla13
(Assignee)

Comment 11

5 years ago
Comment on attachment 598235 [details] [diff] [review]
Ship xpidllex.py and xpidlyacc.py

[Approval Request Comment]
This prevents the SDK from being completely useful (though this can be worked around by giving a --cachedir to the header.py and typelib.py scripts). This only ships two additional files in the SDK and changes nothing else.
Risk to taking this patch (and alternatives if risky): none
String changes made by this patch: none

I'd request approval for esr, but the flag doesn't show up.
Attachment #598235 - Flags: approval-mozilla-beta?
Attachment #598235 - Flags: approval-mozilla-aurora?
Since this landed, my local -inbound tree fails to build on OS X (10.6), with the error:

make[7]: Entering directory `/Users/jonathan/mozdev/mozilla-inbound/obj-ff/xpcom/typelib/xpidl'
/Users/jonathan/mozdev/mozilla-inbound/obj-ff/config/nsinstall -D ../../../dist/sdk/bin/ply
/Users/jonathan/mozdev/mozilla-inbound/obj-ff/config/nsinstall -L /Users/jonathan/mozdev/mozilla-inbound/obj-ff/xpcom/typelib/xpidl -m 755 /Users/jonathan/mozdev/mozilla-inbound/other-licenses/ply/ply/__init__.py /Users/jonathan/mozdev/mozilla-inbound/other-licenses/ply/ply/lex.py /Users/jonathan/mozdev/mozilla-inbound/other-licenses/ply/ply/yacc.py ../../../dist/sdk/bin/ply
make[8]: Entering directory `/Users/jonathan/mozdev/mozilla-inbound/obj-ff/xpcom/typelib/xpidl'
make[8]: *** No rule to make target `../../../xpcom/idl-parser/xpidllex.py', needed by `libs'.  Stop.

Obviously, this isn't affecting tinderbox, as tbpl shows happy builds ... I don't know why it's failing locally for me.

Comment 13

5 years ago
I have the same build failure on Mac OS X 10.7.
(Assignee)

Comment 14

5 years ago
So, after investigation, Josh and Jonathan's problem was leftovers from times when xpidllex.py and xpidlyacc.py were created in the srcdir instead of the objdir.

Comment 15

5 years ago
(In reply to Mike Hommey [:glandium] from comment #14)
> So, after investigation, Josh and Jonathan's problem was leftovers from
> times when xpidllex.py and xpidlyacc.py were created in the srcdir instead
> of the objdir.

Sure? I get the error of Comment #12 even after hg co -C && hg purge. In order to compile m-c I had to backout 013b99c7ae65.
(Assignee)

Comment 16

5 years ago
(In reply to Stefan from comment #15)
> (In reply to Mike Hommey [:glandium] from comment #14)
> > So, after investigation, Josh and Jonathan's problem was leftovers from
> > times when xpidllex.py and xpidlyacc.py were created in the srcdir instead
> > of the objdir.
> 
> Sure? I get the error of Comment #12 even after hg co -C && hg purge. In
> order to compile m-c I had to backout 013b99c7ae65.

rm $objdir/xpcom/idl-parser/*.pyc ; they are hgignored, and hg purge doesn't remove them.
(Assignee)

Comment 17

5 years ago
(In reply to Mike Hommey [:glandium] from comment #16)
> rm $objdir/xpcom/idl-parser/*.pyc ; they are hgignored, and hg purge doesn't
> remove them.

erm, that's rm $srcdir/xpcom/idl-parser/*.pyc
https://hg.mozilla.org/mozilla-central/rev/013b99c7ae65
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED

Updated

5 years ago
Depends on: 729752
Attachment #598235 - Flags: approval-mozilla-beta?
Attachment #598235 - Flags: approval-mozilla-beta+
Attachment #598235 - Flags: approval-mozilla-aurora?
Attachment #598235 - Flags: approval-mozilla-aurora+
tracking-firefox-esr10: --- → 11+
Please see https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for information on landing on mozilla-esr10 before 3/2.
(Assignee)

Comment 20

5 years ago
http://hg.mozilla.org/releases/mozilla-aurora/rev/1f89179f19d7
http://hg.mozilla.org/releases/mozilla-beta/rev/76add95f5ce4

Also landed on the COMM110_2012022204_RELBRANCH by mistake :(
http://hg.mozilla.org/releases/mozilla-beta/rev/5738e8b0a963
(Assignee)

Updated

5 years ago
status-firefox-esr10: affected → checkin-pending
status-firefox11: affected → fixed
status-firefox12: affected → fixed
[Triage comment]

Hi Mike, this bug is being tracked for landing on ESR branch.  Please land patches on http://hg.mozilla.org/releases/mozilla-esr10/ before Thursday March 1, 2012 in order to be ready for go-to-build on Friday March 2, 2012.
(Assignee)

Comment 22

5 years ago
http://hg.mozilla.org/releases/mozilla-esr10/rev/52637db4f742
status-firefox-esr10: checkin-pending → fixed
(In reply to Mike Hommey [:glandium] from comment #14)
> So, after investigation, Josh and Jonathan's problem was leftovers from
> times when xpidllex.py and xpidlyacc.py were created in the srcdir instead
> of the objdir.

Could we add an rm command somewhere during configure or make so that this automatically gets fixed?
(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) (away February 21 to March 17) from comment #23)
> (In reply to Mike Hommey [:glandium] from comment #14)
> > So, after investigation, Josh and Jonathan's problem was leftovers from
> > times when xpidllex.py and xpidlyacc.py were created in the srcdir instead
> > of the objdir.
> 
> Could we add an rm command somewhere during configure or make so that this
> automatically gets fixed?

In principle, I don't think configure or make should be modifying the source tree in any way.
Sure, but in this case they'd just be undoing a modification that shouldn't have happened in the first place.

Comment 26

5 years ago
(In reply to Jonathan Kew (:jfkthame) from comment #24)
> In principle, I don't think configure or make should be modifying the source
> tree in any way.

Sure, if you build outside of the source tree. But building outside the source tree (on which I removed the write permission) fails:

/bin/sh: ../m-c/.mozconfig.out: Permission denied

client.mk:

    136 run_for_side_effects := \
    137   $(shell $(TOPSRCDIR)/$(MOZCONFIG_LOADER) $(TOPSRCDIR) $(TOPSRCDIR)/.mo        zconfig.mk > $(TOPSRCDIR)/.mozconfig.out)
                                   ^

Comment 27

5 years ago
Can someone possibly re-open this bug by changing the bug status?
I got bitten by the same bug (I think), but since its status was fixed, it didn't show up on the first simple search.

I my case, I had to install system's python-ply (lex/yacc tool) first, and run make clean and then recompile everything. I am not sure why, but both the new python tools and make clean was necessary on my Debian [wheezy] system.)

Looks to me that these are all related:

691836 	nor 	-- 	Linu 	nobody 	NEW 	--- 	Build system doesn't correctly
reparse IDL files when the xpidl parser has changed

729752 	nor 	-- 	All 	nobody 	NEW 	--- 	Old .pyc files in srcdir shouldn't
break the build
and this one.
(Assignee)

Comment 28

5 years ago
For what it's worth, "make clean" removes the problematic .pyc files from the source directory because of http://mxr.mozilla.org/mozilla-central/source/js/xpconnect/src/Makefile.in#250
Whiteboard: [qa-]

Comment 29

5 years ago
comment 28: I ran into this bug, didn't think of make clean since I blew away my obj dir but it was still breaking. We should auto-clean these files when building.
Whiteboard: [qa-] → [qa-][read Comment 17 if you're hitting this]

Updated

5 years ago
Duplicate of this bug: 781525

Updated

4 years ago
status1.9.2: unaffected → ---
status-firefox-esr10: fixed → ---
status-firefox10: affected → ---
status-firefox11: fixed → ---
status-firefox12: fixed → ---
status-firefox13: fixed → ---
tracking-firefox-esr10: 11+ → ---
You need to log in before you can comment on or make changes to this bug.