make clean doesn't remove xpidl files

RESOLVED FIXED in mozilla27

Status

defect
RESOLVED FIXED
6 years ago
2 years ago

People

(Reporter: gaston, Assigned: gps)

Tracking

unspecified
mozilla27
x86_64
OpenBSD
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

Since a little while my openbsd clobber builds randomly fail with missing nsISupports.h headers in the path, so that might be a dependency problem ? Usually removing the objdir fixes it but that's not a proper fix. Dunno what's supposed to generate nsISupports.h ?


eg++ -o nsReadableUtils.o -c  -fvisibility=hidden -DMOZ_GLUE_IN_PROGRAM -DMOZILLA_INTERNAL_API -DNO_NSPR_10_SUPPORT -DIMPL_LIBXUL  -I/home/buildslave/mozilla-central-sparc64/build/xpcom/string/src -I. -I../../../dist/include  -I/data/obj/buildslave/m-c/dist/include/nspr -I/data/obj/buildslave/m-c/dist/include/nss      -fPIC  -I/usr/X11R6/include -DMOZILLA_CLIENT -include ../../../mozilla-config.h -MD -MP -MF .deps/nsReadableUtils.o.pp -I/usr/X11R6/include -Wall -Wpointer-arith -Woverloaded-virtual -Werror=return-type -Wtype-limits -Wempty-body -Wsign-compare -Wno-invalid-offsetof -fno-exceptions -fno-strict-aliasing -fno-rtti -fno-exceptions -fno-math-errno -std=gnu++0x -pthread -pipe  -DNDEBUG -DTRIMMED -g -O -fomit-frame-pointer  /home/buildslave/mozilla-central-sparc64/build/xpcom/string/src/nsReadableUtils.cpp
nsString.o
In file included from ../../../dist/include/nsCycleCollectionNoteChild.h:12:0,
                 from ../../../dist/include/nsTArray.h:18,
                 from /home/buildslave/mozilla-central-sparc64/build/xpcom/string/src/nsReadableUtils.cpp:10:
../../../dist/include/nsCycleCollectionTraversalCallback.h:9:25: fatal error: nsISupports.h: No such file or directory
compilation terminated.

This is happening regardless of arch & compiler used.
http://buildbot.rhaalovely.net/builders/mozilla-central-amd64/builds/874 is an occurence from some days ago.. i think it was one of the first hit.
http://buildbot.rhaalovely.net/builders/mozilla-central-sparc64/builds/554 is another from yesterday.

It also affects c-c, see http://buildbot.rhaalovely.net/builders/comm-central-amd64/builds/855

Sometimes, it's on a missing nsIAtom.h:

eg++ -o nsSubstring.o -c -I../../../dist/stl_wrappers -I../../../dist/system_wrappers -include /home/buildslave/comm-central-i386/build/mozilla/config/gcc_hidden.h -DMOZ_GLUE_IN_PROGRAM -DMOZILLA_INTERNAL_API -DNO_NSPR_10_SUPPORT -DIMPL_LIBXUL  -I/home/buildslave/comm-central-i386/build/mozilla/xpcom/string/src -I. -I../../../dist/include  -I/var/buildslave/comm-central-i386/build/obj-i386-unknown-openbsd5.4/mozilla/dist/include/nspr -I/var/buildslave/comm-central-i386/build/obj-i386-unknown-openbsd5.4/mozilla/dist/include/nss      -fPIC  -I/usr/X11R6/include -DMOZILLA_CLIENT -include ../../../mozilla-config.h -MD -MP -MF .deps/nsSubstring.o.pp -I/usr/X11R6/include -Wall -Wpointer-arith -Woverloaded-virtual -Werror=return-type -Wtype-limits -Wempty-body -Wsign-compare -Wno-invalid-offsetof -Wcast-align -fno-exceptions -fno-strict-aliasing -fno-rtti -fno-exceptions -fno-math-errno -std=gnu++0x -pthread -pipe  -DNDEBUG -DTRIMMED -O -fomit-frame-pointer  /home/buildslave/comm-central-i386/build/mozilla/xpcom/string/src/nsSubstring.cpp
In file included from /home/buildslave/comm-central-i386/build/mozilla/xpcom/string/src/nsSubstring.cpp:25:0:
../../../dist/include/nsStaticAtom.h:9:21: fatal error: nsIAtom.h: No such file or directory
compilation terminated.

(from http://buildbot.rhaalovely.net/builders/comm-central-i386/builds/407)
http://buildbot.rhaalovely.net/builders/mozilla-central-amd64/builds/880 is also an occurence of a missing nsIAtom.h
Summary: Random build failure with missing nsISupports.h header → Random build failure with missing nsISupports.h or nsIAtom.h header
More failures... removed the objdirs this time. The usual process is hg pull -u, gmake -f client.mk, gmake package, gmake clean. In case nsI* are sometimes not generated (missing dep ?)

http://buildbot.rhaalovely.net/builders/mozilla-central-sparc64/builds/556
http://buildbot.rhaalovely.net/builders/mozilla-central-amd64/builds/886
http://buildbot.rhaalovely.net/builders/comm-central-amd64/builds/861
Sadly, this also affects aurora:
http://buildbot.rhaalovely.net/builders/mozilla-aurora-amd64/builds/871

ccing some hopefully helpful folks for hints on the possible cause of this :)
Maybe Ehsan has a better idea of what might have regressed it.  It sounds like something the recent flurry of work on #include elimination could have broken.
Flags: needinfo?(ehsan)
Sounds more like the exporting work
Flags: needinfo?(gps)
Yeah this is almost definitely a race condition in the build system.
Component: XPCOM → Build Config
Flags: needinfo?(ehsan)
(In reply to comment #5)
> Maybe Ehsan has a better idea of what might have regressed it.  It sounds like
> something the recent flurry of work on #include elimination could have broken.

Note that could not have possibly happened, since this bug seems to be about the files not being there on the filesystem, it seems.
(In reply to :Ehsan Akhgari (needinfo? me!) from comment #8)
> (In reply to comment #5)
> > Maybe Ehsan has a better idea of what might have regressed it.  It sounds like
> > something the recent flurry of work on #include elimination could have broken.
> 
> Note that could not have possibly happened, since this bug seems to be about
> the files not being there on the filesystem, it seems.

Yeah, when the failure shows up, the offending file is not in the objdir at all, ie not generated.
You are seeing this on current mozilla-central? I could understand nsISupports.h being out of date in dist/includes, but for it to not be present at all is, uh, very surprising.

What do you see in objdir/config/makefiles/xpidl?
Flags: needinfo?(gps)
My other builders are failing hard atm because of #919111 but for aurora from yesterday:
http://buildbot.rhaalovely.net/builders/mozilla-aurora-amd64/builds/873/steps/build/logs/stdio

This is what i have there:

/usr/obj/buildslave-m-a/config/makefiles/xpidl/Makefile

/usr/obj/buildslave-m-a/config/makefiles/xpidl/xpt:
WorkerTest.xpt                  dom_base.xpt                    dom_svg.xpt                     intl.xpt                        necko_websocket.xpt             update.xpt
accessibility.xpt               dom_camera.xpt                  dom_system.xpt                  jar.xpt                         necko_wyciwyg.xpt               uriloader.xpt
alerts.xpt                      dom_canvas.xpt                  dom_telephony.xpt               jsdebugger.xpt                  parentalcontrols.xpt            url-classifier.xpt
appshell.xpt                    dom_contacts.xpt                dom_time.xpt                    jsdownloads.xpt                 pipboot.xpt                     urlformatter.xpt
appstartup.xpt                  dom_core.xpt                    dom_traversal.xpt               jsdservice.xpt                  pipnss.xpt                      webBrowser_core.xpt
autocomplete.xpt                dom_css.xpt                     dom_webspeechrecognition.xpt    jsinspector.xpt                 pippki.xpt                      webbrowserpersist.xpt
autoconfig.xpt                  dom_devicestorage.xpt           dom_webspeechsynth.xpt          layout_base.xpt                 places.xpt                      widget.xpt
browser-feeds.xpt               dom_events.xpt                  dom_xbl.xpt                     layout_printing.xpt             plugin.xpt                      windowds.xpt
browsercompsbase.xpt            dom_file.xpt                    dom_xpath.xpt                   layout_xul.xpt                  pref.xpt                        windowwatcher.xpt
caps.xpt                        dom_gamepad.xpt                 dom_xul.xpt                     layout_xul_tree.xpt             prefetch.xpt                    xpcom_base.xpt
chrome.xpt                      dom_geolocation.xpt             downloads.xpt                   locale.xpt                      profile.xpt                     xpcom_components.xpt
commandhandler.xpt              dom_html.xpt                    editor.xpt                      loginmgr.xpt                    rdf.xpt                         xpcom_ds.xpt
commandlines.xpt                dom_indexeddb.xpt               embed_base.xpt                  lwbrk.xpt                       satchel.xpt                     xpcom_io.xpt
composer.xpt                    dom_json.xpt                    extensions.xpt                  migration.xpt                   saxparser.xpt                   xpcom_system.xpt
content_base.xpt                dom_media.xpt                   exthandler.xpt                  mimetype.xpt                    services-crypto-component.xpt   xpcom_threads.xpt
content_canvas.xpt              dom_messages.xpt                exthelper.xpt                   mozfind.xpt                     sessionstore.xpt                xpcom_xpti.xpt
content_events.xpt              dom_mobilemessage.xpt           fastfind.xpt                    necko.xpt                       shellservice.xpt                xpcomsample.xpt
content_html.xpt                dom_network.xpt                 feeds.xpt                       necko_about.xpt                 shistory.xpt                    xpconnect.xpt
content_htmldoc.xpt             dom_notification.xpt            filepicker.xpt                  necko_cache.xpt                 spellchecker.xpt                xpctest.xpt
content_xslt.xpt                dom_offline.xpt                 find.xpt                        necko_cookie.xpt                startupcache.xpt                xul.xpt
cookie.xpt                      dom_permissionsettings.xpt      fuel.xpt                        necko_dns.xpt                   storage.xpt                     xulapp.xpt
directory.xpt                   dom_power.xpt                   gfx.xpt                         necko_file.xpt                  telemetry.xpt                   xuldoc.xpt
diskspacewatcher.xpt            dom_quota.xpt                   hal.xpt                         necko_ftp.xpt                   test_necko.xpt                  xultmpl.xpt
docshell.xpt                    dom_range.xpt                   html5.xpt                       necko_http.xpt                  toolkitprofile.xpt              zipwriter.xpt
dom.xpt                         dom_settings.xpt                htmlparser.xpt                  necko_ipc.xpt                   toolkitremote.xpt
dom_activities.xpt              dom_sidebar.xpt                 identity.xpt                    necko_res.xpt                   txmgr.xpt
dom_alarm.xpt                   dom_smil.xpt                    imgicon.xpt                     necko_socket.xpt                txtsvc.xpt
dom_apps.xpt                    dom_storage.xpt                 imglib2.xpt                     necko_strconv.xpt               uconv.xpt
dom_audiochannel.xpt            dom_stylesheets.xpt             inspector.xpt                   necko_viewsource.xpt            unicharutil.xpt


And the list of nsI* headers in dist/include:
[21:18] dawn:/usr/obj/buildslave-m-a/dist/include/ $ls nsI*
nsIAccessibilityService.h@            nsIDTD.h@                             nsIJSNativeInitializer.h@             nsIPrivateTextEvent.h@                nsIStyleRuleProcessor.h@
nsIAllocator.h@                       nsIDateTimeFormat.h@                  nsILanguageAtomService.h@             nsIPrivateTextRange.h@                nsIStyleSheet.h@
nsIAnonymousContentCreator.h@         nsIDeviceContextSpec.h@               nsILayoutDebugger.h@                  nsIRDFContentSink.h@                  nsIStyleSheetLinkingElement.h@
nsIAppStartupNotifier.h@              nsIDocument.h@                        nsILayoutHistoryState.h@              nsIRadioGroupContainer.h@             nsISupportsBase.h@
nsIAttribute.h@                       nsIDocumentInlines.h@                 nsILineBreaker.h@                     nsIRadioVisitor.h@                    nsISupportsImpl.h@
nsIAudioChannelAgent.h                nsIDocumentObserver.h@                nsILineIterator.h@                    nsIReflowCallback.h@                  nsISupportsObsolete.h@
nsICSSDeclaration.h@                  nsIDocumentTransformer.h@             nsILinkHandler.h@                     nsIRollupListener.h@                  nsISupportsUtils.h@
nsICSSLoaderObserver.h@               nsIFileStorage.h@                     nsIListControlFrame.h@                nsISMILAttr.h@                        nsITableCellLayout.h@
nsICSSPseudoComparator.h@             nsIForm.h@                            nsILocalStore.h@                      nsISMILType.h@                        nsITextControlElement.h@
nsICSSRuleList.h@                     nsIFormControl.h@                     nsIMutationObserver.h@                nsIScriptContext.h@                   nsITextControlFrame.h@
nsICSSStyleRuleDOMWrapper.h@          nsIFormControlFrame.h@                nsINIParser.h@                        nsIScriptElement.h@                   nsITextService.h@
nsICachedFileDescriptorListener.h@    nsIFormProcessor.h@                   nsINameSpaceManager.h@                nsIScriptExternalNameSet.h@           nsITextServicesDocument.h@
nsICanvasElementExternal.h@           nsIFragmentContentSink.h@             nsINativeKeyBindings.h@               nsIScriptGlobalObject.h@              nsITheme.h@
nsICanvasRenderingContextInternal.h@  nsIFrame.h@                           nsINode.h@                            nsIScriptGlobalObjectOwner.h@         nsITokenizer.h@
nsICaseConversion.h@                  nsIFrameTraversal.h@                  nsINodeInfo.h@                        nsIScriptNameSpaceManager.h@          nsIUGenCategory.h@
nsICharsetDetectionObserver.h@        nsIFrameUtil.h@                       nsINodeList.h@                        nsIScriptObjectPrincipal.h@           nsIUnicodeDecoder.h@
nsICharsetDetector.h@                 nsIGlobalObject.h@                    nsIOS2Locale.h@                       nsIScriptTimeoutHandler.h@            nsIUnicodeEncoder.h@
nsIClassInfoImpl.h@                   nsIGridPart.h@                        nsIObjectFrame.h@                     nsIScrollPositionListener.h@          nsIWeakReferenceUtils.h@
nsIComboboxControlFrame.h@            nsIHTMLCollection.h@                  nsIOfflineStorage.h@                  nsIScrollableFrame.h@                 nsIWebShellServices.h@
nsIConstraintValidation.h@            nsIHTMLContentSink.h@                 nsIPageSequenceFrame.h@               nsIScrollbarMediator.h@               nsIWidget.h@
nsIContent.h@                         nsIHTMLDocument.h@                    nsIParser.h@                          nsIScrollbarOwner.h@                  nsIWidgetListener.h@
nsIContentIterator.h@                 nsIID.h@                              nsIParserService.h@                   nsISelectControlFrame.h@              nsIWordBreaker.h@
nsIContentSerializer.h@               nsIIPCSerializableInputStream.h@      nsIPercentHeightObserver.h@           nsISizeOf.h@                          nsIXMLContentSink.h@
nsIContentSink.h@                     nsIIPCSerializableURI.h@              nsIPlatformCharset.h@                 nsISpellChecker.h@                    nsIXULDocument.h@
nsID.h@                               nsIImageToPixbuf.h@                   nsIPluginWidget.h@                    nsIStatefulFrame.h@                   nsImageLoadingContent.h@
nsIDOMClassInfo.h@                    nsIInterfaceRequestorUtils.h@         nsIPresShell.h@                       nsIStringCharsetDetector.h@           nsInterfaceHashtable.h@
nsIDOMScriptObjectFactory.h@          nsIJSEventListener.h@                 nsIPrintDialogService.h@              nsIStyleRule.h@                       nsInterfaceRequestorAgg.h@
I believe there was a bug in the build rules impacting Windows. But that shouldn't have impacted any platform with symlinks.

Are you doing any kind of manual clobber, such as deleting bits of dist/include? We don't have explicit rules to install XPIDL-deleted dist/include/.h files if they are deleted (we assume the build system behaves a certain way, which it should as long as you are building through client.mk). 

Do you have bug 908977 / ba4acc701c7e in your tree? That should have fixed lingering build dependency issues.
(In reply to Gregory Szorc [:gps] from comment #12)
> I believe there was a bug in the build rules impacting Windows. But that
> shouldn't have impacted any platform with symlinks.
> 
> Are you doing any kind of manual clobber, such as deleting bits of
> dist/include? We don't have explicit rules to install XPIDL-deleted
> dist/include/.h files if they are deleted (we assume the build system
> behaves a certain way, which it should as long as you are building through
> client.mk). 

The buildslave does this every 24h (you can see the steps on http://buildbot.rhaalovely.net/builders/mozilla-aurora-amd64/builds/873):
- hg qpop -a (in case i have patches to fix build issues in my queue)
- hg pull -u
- hg qpush -a
- gmake -f client.mk configure
- gmake -f client.mk
- gmake -C objdir package
- gmake -f client.mk clean

Of course, all the steps are done only if the previous succeeds. That means the clean step is done only if everything went fine, so that i can have a look at the objdir. In all the previous occurences of that bug, manually rm -Rf'ing the objdir 'fixed' the issue for a day or two, then it usually happens again. SO no, i dont manually purge the objdir, it's an all or nothing thing.

> Do you have bug 908977 / ba4acc701c7e in your tree? That should have fixed
> lingering build dependency issues.

The random failures started happening something like 15 days ago, and now happens on aurora & central, so yes i have that changeset.
|make clean| will blow out dist/. However, it won't delete the .xpt files from config/makefiles/xpidl. As a result, the build system thinks the .h in dist/include are up to date and doesn't reinstall them. We should delete the .xpt files during |make clean|.
Summary: Random build failure with missing nsISupports.h or nsIAtom.h header → make clean doesn't remove xpidl files
(In reply to Gregory Szorc [:gps] from comment #14)
> |make clean| will blow out dist/. However, it won't delete the .xpt files
> from config/makefiles/xpidl. As a result, the build system thinks the .h in
> dist/include are up to date and doesn't reinstall them. We should delete the
> .xpt files during |make clean|.

so you mean i should rather replace my last step by make clobber ?
what recently changed in that area that could trigger that new failure ?
(In reply to Landry Breuil (:gaston) from comment #15)
> (In reply to Gregory Szorc [:gps] from comment #14)
> > |make clean| will blow out dist/. However, it won't delete the .xpt files
> > from config/makefiles/xpidl. As a result, the build system thinks the .h in
> > dist/include are up to date and doesn't reinstall them. We should delete the
> > .xpt files during |make clean|.
> 
> so you mean i should rather replace my last step by make clobber ?
> what recently changed in that area that could trigger that new failure ?

I think |make clobber| is busted. Use |mach clobber| or rm -rf the objdir.

We should fix |make clean|. This is fallout from bug 850380.
Depends on: 850380
Another interesting different one:
http://buildbot.rhaalovely.net/builders/mozilla-central-amd64/builds/891

In file included from /var/buildslave-mozilla/mozilla-central-amd64/build/xpcom/base/AvailableMemoryTracker.cpp:22:
../../dist/include/mozilla/Preferences.h:13:10: fatal error: 'nsIPrefService.h' file not found
#include "nsIPrefService.h"
         ^
1 error generated.
gmake[5]: *** [AvailableMemoryTracker.o] Error 1
gmake[5]: *** Waiting for unfinished jobs....
In file included from /var/buildslave-mozilla/mozilla-central-amd64/build/xpcom/base/CycleCollectedJSRuntime.cpp:60:
In file included from ../../dist/include/mozilla/dom/BindingUtils.h:14:
In file included from ../../dist/include/mozilla/dom/CallbackObject.h:32:
../../dist/include/xpcpublic.h:17:10: fatal error: 'nsIURI.h' file not found
#include "nsIURI.h"
Funky new one too.. 
http://buildbot.rhaalovely.net/builders/mozilla-central-sparc64/builds/561

In file included from /home/buildslave/mozilla-central-sparc64/build/xpcom/base/AvailableMemoryTracker.cpp:22:0:
../../dist/include/mozilla/Preferences.h:13:28: fatal error: nsIPrefService.h: No such file or directory
compilation terminated.

I'll remove all objdirs and switch to mach clobber on my builders until the issue is fixed.
This bug still affects my c-c builder where i cant really use mach clobber...

Even doing gmake -f client.mk clean + rm -Rf objdir/dist* after a successful build, the next one fails.

http://buildbot.rhaalovely.net/builders/comm-central-amd64/builds/880
I hate having to redefine the targets like this, but it's the simplest
patch. At some point we should merge the two xpidl Makefile.in together.
Attachment #815635 - Flags: review?(mh+mozilla)
Assignee: nobody → gps
Attachment #815635 - Flags: review?(mh+mozilla) → review+
https://hg.mozilla.org/integration/mozilla-inbound/rev/19978aab0190
Status: NEW → ASSIGNED
Flags: in-testsuite-
https://hg.mozilla.org/mozilla-central/rev/19978aab0190
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla27
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.