Open Bug 104642 Opened 19 years ago Updated 9 months ago
can't regenerate configure with autoconf 2
I upgraded to autoconf 2.52 recently, and now I can't generate configure; running autoconf gives me: build/autoconf/altoptions.m4:140: error: m4_defn: undefined macro: _m4_divert_diversion build/autoconf/altoptions.m4:137: MOZ_READ_MOZCONFIG is expanded from... build/autoconf/altoptions.m4:140: the top level Any ideas?
Obviously, 2.5 doesn't like our custom m4 macros. I haven't played with 2.5x yet so I have no idea how to quick fix this.
Priority: -- → P4
Target Milestone: --- → mozilla1.0
I managed to get autoconf to generate a configure file, but then I ran into the problem that AC_PROG_CC now only tests for the object extention the first time - ie in the cros_compile block. I posted this to the autoconf-bugs list, and got the response: > That's correct, using conditional checking with Autoconf is a very bad > idea. You should never do that. Always check, and perform > conditional actions, but don't make conditional checks. So, suggestions?
Target Milestone: mozilla1.0 → mozilla0.9.9
I think Corinna summed up the situation perfectly at: http://cygwin.com/ml/cygwin-announce/2001/msg00177.html . Outside of our own 1.0 deadline issues, I can't think of any reason that we'd want to go thru the trouble of making the jump to 2.5x right now.
Target Milestone: mozilla0.9.9 → Future
So, it's been 6 months since this was looked at, and quite a few linux vendors are now shipping the newer autotools (but not, I think, RedHat). Is this going to require a drastic overhaul of the configure script (and therefore be unlikely to happen anytime soon) or just some localized tweaking? To my shame, my knowledge of m4 is almost non-existant, so I'm hesitant to wade into the files, everything in mozilla/config is foreign territory to me :-)
Yes, this is going to require a drastic overhaul of the script. The autoconf maintainer's insistence upon not supporting conditional checks (meaning that you can only check for a particular feature once) means that we'll need to completely rethink how we handle the cross-compile checking and the checks for libXt's dependency upon libSM & libICE at the very least. At the bottom of configure.in, we pass additional arguments to nspr & ldap's configures when they are run. Last time I looked into it, there was no way to do that in autoconf 2.5x. AC_CONFIG_SUBDIRS changed so the hack that we're using for 2.1x doesn't work. I've seen a couple of other people ask about the same question on the autoconf mailing with no positive responses. So far, I haven't seen any feature of autoconf 2.5x that makes it worth the effort of switching. They have fixed one of the major performance issues with generating a new configure so that's one strike removed at least. Since the version of autoconf required (2.12+) is still available for download and this doesn't affect the end product nor does it affect development unless you're working on the build system, I'm not seeing the need to force the upgrade.
Thanks for confirming my fears cls: the problem is, I do I have to work on the build system .. but ours, not yours. :-) We (Crocodile Clips) add various subdirectories into the main build (thankfully, no configure changes are required at the moment, but doubtless that day is fast approaching). So we modify allmakefiles.sh, and hence need to be able to regenerate. Right now, this is just an annoyance (we had to use RedHat instead of Suse, and I can't upgrade to automake 1.4 under OS-X becuase it needs autoconf 2.5) but I would like to avoid getting bitten by this when I do an OS upgrade and our build system is dead in the water.
FWIW, as long as you don't touch configure.in, aclocal.m4 or build/autoconf/*.m4, you don't need to run autoconf. Configure reads allmakefiles.sh when configure is run and we have build rules to regenerate sub-makefiles on the fly.
Mass reassign to new default build assignee
Assignee: seawood → mozbugs-build
Priority: P4 → --
Mass reassign of Build/Config bugs to Leaf.
Assignee: mozbugs-build → leaf
Target Milestone: Future → ---
http://www.mozilla.org/build/unix-details.html#s3 says autoconf-2.12 is required, yet "configure" in the official source tar distribution for mozilla-1.5 contains a remark that it was built with autoconf-2.13. Please update the unix install instructions.
Since we're up to revision around 1500 to configure.in alone, perhaps we should start working on updating it and the m4 macros, etc to work with autoconf tools newer than 5+ years old. It's impossible for people to test configure.in changes before committing them, or to review/test patches that include it. Double so because people are saying that it's not policy to include patches to configure in a patch.
Ugh. The sky is not falling. What does the revision number of configure.in indicate besides that it's frequently modified? It may be inconvenient for some to test configure.in changes but it's not even close to impossible. autoconf 2.13 only has a dependency upon m4 (and optionally perl) and doesn't affect anything else so anyone could install it locally in their homedir if their system didn't provide the package. IMO, that's no different than needing to install any of the other build prerequisites (glib, libIDL, etc). For the signficant effort required, how does this move benefit us besides being able to run 'autoconf' out of the box on some machines (which we can arguably already do)? IMO, the real showstopper isn't our macros or cross-compiler checks, which can be rewritten with some effort. It's the fact that you can't add additional arguments when calling the sub-configures. Because of that, either a) the sub-configures need to use the defaults as required by the toplevel configure script or b) we need to invoke the sub-configures independently of the toplevel configure script. (a) is unlikely given that the sub-configures are for independent projects (NSPR & LDAP) and the arguments will change depending upon the build configuration. With (b), we run into problems because we want use the same environment setup that the toplevel configure was run against and we want to use the output of the toplevel configure to choose the arguments to pass to the sub-configures. (Hmm, that might actually be doable with some additional makefiles & client.mk hacks but it'll definitely be more complex than what we have now.)
OS: Linux → All
Product: Mozilla Application Suite → Core
Hardware: PC → All
I would like to avoid the client.mk hackery for this, preserving the ability to run ../configure --enable-application=browser; make; and have firefox come out the other end. autoconf-2.13 remains a good solution and it is available as an RPM, so unless somebody is signing up to do the autoconf2.5x work this is really WONTFIX.
autoconf has moved on from 2.13 and modern versions have some useful features. There is certainly no fundamental problem implementing the configure syntax you want with autoconf 2.5x. Why should this be WONTFIX just because there's no one tackling it Right Now?
Not every distribution is capable of having multiple versions of autoconf at the same time ( non deb or rpm ones like mine aren't). The only autoconf version I have access to is 2.60
Mass re-assign of bugs that aren't on the build team radar, so bugs assigned to email@example.com reflects reality. If there is a bug you really think we need to be looking at, please *email* firstname.lastname@example.org with a bug number and explanation.
Assignee: build → nobody
At some point we are going to run into the problem that autoconf-2.13 is going to require more and more hacks to work with newer platforms and compilers, at which point it may become necessary to switch. We (mozldap) have already run into this problem with HP/ux 11.23 ipf 64bit and SunOS 10 amd64. Are there any plans at all to move to a newer version of autoconf?
Here's my current working patch to make it work with autoconf 2.61. It's not cleaned up and documented yet. Feel free to ask your questions :) (BTW: dev-builds@ is probably an good place for discussions).
Enrico: do you have a patch for mozilla/nsprpub/configure.in? Thanks.
(In reply to comment #20) > Enrico: do you have a patch for mozilla/nsprpub/configure.in? Thanks. Not yet. I'm using the system-nspr which gentoo had cleanly installed. BTW: I'm working on an patch to kick off the whole third-party libs, including nspr, etc. Let's talk on dev-buils on that :)
(In reply to comment #0) > build/autoconf/altoptions.m4:140: error: m4_defn: undefined macro: > _m4_divert_diversion > build/autoconf/altoptions.m4:137: MOZ_READ_MOZCONFIG is expanded from... > build/autoconf/altoptions.m4:140: the top level That's only one of the problems in configure.in. I did not find an real solution for that yet (my research showed up that it's probably an autoconf-internal issue), but I worked around it by simply writing the macro code where it is actually called. Seems it's just used once. There were a lot of other issues. You may want to try my patch.
Hi, (In reply to comment #5) > need to completely rethink how we handle the cross-compile checking Simply not using **** macros like AC_TRY_RUN ? Or not assuming we can run anyting at all ? Move out shipped build tools to separate packages ? > and the checks for libXt's dependency upon libSM & libICE at the very least. pkg-config ? > At the bottom of configure.in, we pass additional arguments to nspr & ldap's > configures when they are run. What about simply kicking external libraries off the tree ? At least nspr can exist completely for it's own and mozilla be built an external libspr. I'd consider this is as stable enough, since Gentoo is doing (fully unmasked) for at least a year. > Last time I looked into it, there was no way to do that in autoconf 2.5x. We're now at 2.61. Maybe you're simply not up to date. I've successfully ported mozilla to autoconf-2.61, and it *does* pass the variables properly. > So far, I haven't seen any feature of autoconf 2.5x that makes it worth the > effort of switching. For me that's not the key. I'm using the recent 2.61, and it works quite well. I don't see why I should have to maintain the really unncessary complexity w/ multiple autoconf versions, just for one package. Mozilla ist just one package of thousands. > Since the version of autoconf required (2.12+) is still available for > download and this doesn't affect the end product nor does it affect > development unless you're working on the build system, If *does* affect, really much things. For example older autoconf was quiite unsuable for clean crosscompiling - if it was possible, it always was a big mess. Ah, and it was broken on many oder sides too, ie. not being able to properly call grep, so detect gcc, etc, etc. Things have become much better now (of course still far away from being perfect). cu
>> Last time I looked into it, there was no way to do that in autoconf 2.5x. > We're now at 2.61. Maybe you're simply not up to date. Maybe you should realize that you're responding to a comment that was made almost five *years* ago. 2.61 was definitely not around then. Neither was pkg-config/auto* support for X11/Xorg. IIRC, NSPR was just starting to be included in linux distributions as a standalone product. Contrary to your statement, many packages were able to cross-compile using autoconf 2.13 without a problem. And five years ago, Mozilla was not the only package that refused to make the incompatible jump to autoconf 2.5x. Add it all up and it wasn't worth the effort for mozilla to make the jump back then. But a lot has changed in five years. While I think it's good that someone's revisiting this issue, you should, at the very least, try to understand that things were significantly different back then. Not to mention that there's more involved here than just being able to run 'autoconf' without an error. There are other constraints, some non-technical, that also need to be addressed. For now, let's ignore the fact that autoconf isn't required to build mozilla & that installing autoconf-2.13 locally is orders of magnitude easier than compiling mozilla itself. If you're serious about getting this patch in (and I have *zero* to do with that now), then you probably need to address the following issues: * Requires autoconf 2.61 - This may be a problem as the official build machines may not have the most recent version of autoconf. FC6 ships with 2.59. * Requires external version of NSPR - Unless product management has changed its stance, mozilla has to be able to build against the in tree versions of NSPR, NSS & LDAP not to mention zlib, png, expat, jpeg, cairo, xpidl, etc. - This means that configure.in needs to retain the ability to add options to sub-configure calls. * Cross-compiling - You mentioned that there's a way to do "clean" cross-compiling with 2.61 and I'm sure the patched configure.in isn't it. - It's not clear from the patch that cross-compiling would still work. The current code explicitly calls AC_CHECK_PROGS to look for potential host compilers. I know that with autoconf 2.5x you pass that list directly to AC_PROG_CC but in earlier versions of autoconf 2.5x, you could only call AC_PROG_CC once. * X11 checks - Using pkg-config for checks means that you're bumping the earliest required version of X11 that you can build against. This may be problematic for systems that do not use Xorg's X11 or use an older version (e.g., solaris).
(In reply to comment #24) Now a few month went by since I posted my patch, which works perfectly good for me. I didn't read anything new about it, ie. if it now went in. The thread @ -dev-builds dried out month ago. > Contrary to your statement, many packages were able to cross-compile > using autoconf 2.13 without a problem. Well, I've done quite a lot in that area. Recent autotools had become much better, but 2.13 was heavily broken. Still many packages are broken, ie. all of those relying on macros like AC_TRY_RUN or those supplying their own build tools w/o proper separation between build and target system. > Not to mention that there's more involved here than just being able to > run 'autoconf' without an error. There are other constraints, some > non-technical, that also need to be addressed. Which 'non-technical' constraints ? > * Requires external version of NSPR > - Unless product management has changed its stance, mozilla has to be able > to build against the in tree versions of NSPR, NSS & LDAP not to mention zlib, > png, expat, jpeg, cairo, xpidl, etc. I never understood why some application ships dozens of external libs. Supporting unmaintained and heavily broken old systems is not an valid reason for me. I'm in the process of kicking them off in my own branch. > * Cross-compiling > - You mentioned that there's a way to do "clean" cross-compiling with 2.61 > and I'm sure the patched configure.in isn't it. > - It's not clear from the patch that cross-compiling would still work. The > current code explicitly calls AC_CHECK_PROGS to look for potential host > compilers. I know that with autoconf 2.5x you pass that list directly to > AC_PROG_CC but in earlier versions of autoconf 2.5x, you could only call > AC_PROG_CC once. Well, my patch isn't perfect. Most comes from autoupdate. Suggestions/corrections are always appreciated. > * X11 checks > - Using pkg-config for checks means that you're bumping the earliest required > version of X11 that you can build against. This may be problematic for systems > that do not use Xorg's X11 or use an older version (e.g., solaris). Someone could write an little adapter package which checks for X11 libs and installs pkg-config descriptors for those systems. That's not an issue which should be handled within the moz application. cu
> Now a few month went by since I posted my patch, which works perfectly good > for me. I didn't read anything new about it, ie. if it now went in. The thread > @ -dev-builds dried out month ago. The patch may work for you but it won't work for the tinderboxes unless someone goes in and upgrades their autoconf to 2.61; not to mention the other unresolved issues. You simply can't check in a patch like this that doesn't even work on the reference platforms without some serious coordination with the mozilla.org staff. You should read http://www.mozilla.org/hacking/ . > Which 'non-technical' constraints ? Such as the general expectation/requirement that Mozilla being able to build "out of the box" on several reference platforms. (IMO, it's that's not a technical requirement but a political one.) If you want to use the latest version of autoconf, you're going to violate that requirement. > I never understood why some application ships dozens of external libs. The set of external libs/tools needed to build on win32 & mac is minimal. On Linux and other free OSes, we're used to things being more independent. But Mozilla has historically taken the lowest common denominator approach when it comes to dependencies so 3rd party libs that are not shipped with win32 or mac usually wind up being bundled with Mozilla. > Someone could write an little adapter package which checks for X11 libs > and installs pkg-config descriptors for those systems. Until you write that wrapper, I think everyone else will stick with the existing X11 checks. > That's not an issue which should be handled within the moz application. That's debatable. What is not debatable is the fact that you wish to replace something that works for a lot of people with something that may work for some and definitely will not work for others. What are we gaining by disrupting people's existing development process? The chance to use bleeding edge tools? IMO, moving to a newer version of autoconf should have the same level of impact on the development process as autoconf currently has on the final product: next to nothing. The patch should be rewritten to minimize disruption according to the current Mozilla development guidelines and not what the optimal situation for one developer would be...unless we're going to make that the standard for all.
No longer depends on: 409152
Duplicate of this bug: 409152
On my Fedora 8 system (autoconf 2.61) it *still* gives the same problem for building lightning 0.8. It's been 6 years and counting...
udo: Comments like that are not useful. We know it's not fixed, that's why this bug is still open. If you read cls's comments, you will see the reasons why this is not being fixed. autoconf 2.13 is still widely available, so there is very little impact on users, especially since you only need it if you are hacking on configure.in.
This is only a minor annoyance. The only issue is that some distributions don't have autoconf 2.13 (we only have 2.61 here) I had to package autoconf-2.13 locally only for mozilla and I keep alternating autoconf installations. But since this is Linux, it is rather easy to package and install the version I want regardless of the distribution as long as upstream autoconf-2.13 sources are still available. This is because most Linux packaging formats are relatively easy to learn. I myself was pretty annoyed by this bug two years ago but I have learned to live with it now and it no longer bothers me. My suggestion is to keep the bug opened. Even if there is no near solution, that doesn't mean someone in the future won't get this fixed. udo, if you are a packager and you need to create/use patches, I would suggest you use autoconf-2.13 locally and then when diffing your patches, include changes to both configure.in and configure files in your patch. This means you no longer need to ran autoconf-2.13 in the build scripts you use to create Fedora rpms.
Old tools are not easily found on my distro and perhaps not on other distro's. See http://fr2.rpmfind.net/linux/rpm2html/search.php?query=autoconf&submit=Search+...&system=fedora&arch= So I'll have to dig to find stuff that works and has the right version, arch, etc. (yes there *is* stuff, but this is to illustrate) All this effort just because the build environment has not been updated for 6 years. What more useful can I state? Please state the default and supported build environment more clearly in the docs, on the website, etc? Please consider updating to versions of tools that are more common? (not necessarily the newest, latest ones but ones that do work in a similar way as the current distro versions) Because x86_64 is not a `major platform` for lightning developers I had to try to build it myself, so I ran into various issues, of which this one is absolutely related to your work.
I don't know if anyone is looking at this right now and I'm certainly not proposing to do it myself, but I have a suggestion: One of the obstacles is that in newer autoconf there is no support for changing the arguments passed to subdirectory configure scripts by AC_CONFIG_SUBDIRS. Well, why not run the subdirectory configures from the top-level generated Makefile, instead? You can do CONFIGARGS="$ac_configure_args" AC_SUBST([CONFIGARGS]) to make the top-level configure script's arguments accessible to the generated Makefile, and then you have all of GNU Make's facilities for slicing and dicing space-separated lists available. Be aware though that autoconf 2.5x is quite aggressive about single-quoting individual arguments in that list.
Anyone recognized my (meanwhile >1yr old) patch ?
BTW: the m.o policies forced me to do my own fork :( Everyone how's interested in it, feel free to send me a mail ...
Enrico: you have never addressed cls' points from comment 24 and comment 26. Your patch is unacceptable as it stands. I'm sorry if that doesn't work for your purposes, but you will need better justification for replacing something that is working with something that may or may not work for everyone.
Comment on attachment 259542 [details] [diff] [review] unacceptable patch that breaks cross compiling, in tree builds, etc. i'll be helpful and acknowledge your patch.
An update... Just built an workable ff with autoconf 2.6x (come with Fedora 9 x64 package) and it looks OK to me...
Calvin Liu: With or without the patch in this bug?
No. Without the patch.
(In reply to comment 37) Perhaps in Fedora /usr/bin/autoconf is a script that detects the appropriate version to use instead of always referring to the latest version? Debian and Ubuntu do have such a script.
On Mac OS X 10.6 Darwin 10.0.0, gcc 4.2.1 (Apple Inc. build 5646) autoconf (GNU Autoconf) 2.61 / macports 1.8.0 Built FireFox 3.7a2pre2/Gecko 193 fine. No patch.
If you built with client.mk, it runs autoconf 2.13 by default.
Even Slackware uses autoconf 2.6x for 4 years now. If building mozilla (when checking out with HG) still needs 2.13 from 1999, that is really ugly. So using 2.68 doesn't work for me: Adding client.mk options from /usr/local/TB-hg/.mozconfig: MOZ_OBJDIR=$(TOPSRCDIR)/tbird-bin MOZ_MAKE_FLAGS=-j3 AUTOCONF=autoconf make: Entering directory `/usr/local/TB-hg' Generating /usr/local/TB-hg/configure using autoconf cd /usr/local/TB-hg; autoconf mozilla/build/autoconf/acwinpaths.m4:44: error: defn: undefined macro: AC_OUTPUT_FILES mozilla/build/autoconf/acwinpaths.m4:44: the top level autom4te: /usr/bin/m4 failed with exit status: 1 make: *** [/usr/local/TB-hg/configure] Error 1 make: Leaving directory `/usr/local/TB-hg' make: *** [/usr/local/TB-hg/tbird-bin/Makefile] Error 2 Are there any articles on how to install 2.13 alongside existing installations in distros?
Ok, to answer my question, autoconf 2.13 can be compiled from source according to the instructions at https://developer.mozilla.org/en/Linux_Build_Prerequisites .
As mentioned above and in other places online, autoconf >2.13 removed some features that would make it quite painful to move our environment to, especially given all the across-compiling and different OS versions it needs to run on.
(In reply to :aceman from comment #43) [...] > Are there any articles on how to install 2.13 alongside existing > installations in distros? (In reply to :aceman from comment #44) > Ok, to answer my question, autoconf 2.13 can be compiled from source > according to the instructions at > https://developer.mozilla.org/en/Linux_Build_Prerequisites . Yeah, that is indeed where you learn that quite a number of distros include a package with autoconf 2.13 pre-built (and called autoconf213 or some such to avoid conflict with the "newer" autoconf that you may install in parallel).
Folks, this is ridiculous ! The issue could have been resolved for about 5 years now. (Looks like mozilla development pratically stopped in favour of silly talking years ago). Anyone still wondering why people massively moving away from mozilla in recent years ? ;-o
Folks, this is ridiculous ! The issue could have been resolved for about 5 years now. (Looks like mozilla development pratically stopped in favour of silly talking years ago). Anyone still wondering why people massively moving away from mozilla in recent years ? ;-o
(In reply to Enrico Weigelt from comment #48) > Folks, this is ridiculous ! > The issue could have been resolved for about 5 years now. > (Looks like mozilla development pratically stopped in favour of silly > talking years ago). > > Anyone still wondering why people massively moving away from mozilla > in recent years ? ;-o If this issue 'could have been resolved', do you not think the devs here would've done that? On the basis of cls' comments, the difficulties in moving to 2.6x or whatever is still there. As Ted mentioned, you still haven't addressed cls' comments. Have you progressed much with the patch you proposed awhile back?
I will bet you one million dollars that the reason "why people massively moving away from mozilla in recent years" has nothing to do with our continued dependence on autoconf 2.13.
Heh, even though I am one of the voters in this bug, I agree with comment 50. The dependence just raises the bar for new developers entering into the mozilla dev process (like me).
(In reply to Enrico Weigelt from comment #48) > The issue could have been resolved for about 5 years now. Could it? Great! You just forgot to attach that patch then, right?
I'm just adding myself to get updates. I was trying to build OpenKomodo on Ubuntu 12.10, and received Mercurial errors. OpenKomodo uses Xulrunner from the Firefox trunk. I guess I'll have to find the old autoconf. Strangly, I didn't have this problem on Windows 7 when building a 64bit SeaMonkey. I was using MSVC9 at the time. Maybe I should try to figure out how to build the project in Qt Creator.
Mozillabuild comes with autoconf 2.13 which is why you didn't have a problem on Windows.
Gregory, I think it would be useful to future trippers-over this bug if you wrote something here about your plans for the build system, in particular whether & on what time frame you are contemplating getting rid of Autoconf altogether.
autoconf/configure is its own little beast. It is outside the scope of the drastic moz.build refactoring we're currently doing. That being said, I know a few people have opinions. Mike Hommey probably has more than anyone.
Debian is now talking about dropping its autoconf2.13 package from unstable (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=752291 and https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=753170 ) which will be, at least, an inconvenience for people doing Firefox development on Debian or Ubuntu. Given that, paging Mike Hommey for the opinions that Greg suggested he might have (a year and a half ago, aheh)
(In reply to Zack Weinberg (:zwol) from comment #57) > Debian is now talking about dropping its autoconf2.13 package from unstable > (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=752291 and > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=753170 ) which will be, at > least, an inconvenience for people doing Firefox development on Debian or > Ubuntu. FWIW, I don't think that's going to happen. I have a long term goal to remove the need for autoconf for gecko entirely, but it's still far.
A good alternative is CMake. It handles submodules and runs on Linux, OSX and Windows. http://www.cmake.org/download/
We're not using CMake.
(In reply to Gregory Szorc [:gps] from comment #61) > We're not using CMake. Fair enough. It would have required a massive rewrite of the build scripts anyway.
You need to log in before you can comment on or make changes to this bug.