455813 - Windows PGO builds fail when --disable-tests is set

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

With tests disabled, I get the following error when trying to do a PGO-enabled build with VC8SP1.

OBJDIR=/c/mozbuild/mozilla/objdir-fx python /c/mozbuild/mozilla/objdir-fx/_profile/pgo/profileserver.py
Traceback (most recent call last):
  File "c:/mozbuild/mozilla/objdir-fx/_profile/pgo/profileserver.py", line 40, in <module>
    PROFILE_DIRECTORY, ())
  File "c:\mozbuild\mozilla\objdir-fx\_profile\pgo\automation.py", line 389, in runApp
    certificateStatus = fillCertificateDB(profileDir)
  File "c:\mozbuild\mozilla\objdir-fx\_profile\pgo\automation.py", line 360, in fillCertificateDB
    status = Process(certutil, ["-N", "-d", profileDir, "-f", pwfilePath], environment()).wait()
  File "c:\mozbuild\mozilla\objdir-fx\_profile\pgo\automation.py", line 82, in __init__
    stdin = inputfile)
  File "c:\mozbuild\python\lib\subprocess.py", line 593, in __init__
    errread, errwrite)
  File "c:\mozbuild\python\lib\subprocess.py", line 815, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
make: *** [profiledbuild] Error 1

Comment 1

[(not currently active) Ted Mielczarek]

Right, so the PGO script uses automation.py, which now expects to have certutil available, which only gets built enable-tests, so this is busted.

Comment 2

[Honza Bambas (:mayhemer)]

Hmm... It looks like we need an option to turn of reference to ssl support when build is not tests-enabled. Other option (IMHO better) is to build certutil/pk12util also when this is PGO build.

Which option is more convenient?

Comment 3

[(not currently active) Ted Mielczarek]

You can't detect if it's a PGO build, so you'd have to build it unconditionally, but I'm ok with either approach.

Comment 4

[Honza Bambas (:mayhemer)]

And what about MOZ_PROFILE_USE ?

Comment 5

[(not currently active) Ted Mielczarek]

That gets set after the script is run (in the second pass). Ok, I lied, there is MOZ_PROFILE_GENERATE, but in general, we should make automation.py work even if --disable-tests is specified.

Comment 6

[Honza Bambas (:mayhemer)]

Attachment: fix v1 [Backout: Comment 10]

With this fix "make -f client.mk profiledbuild" passes. Tested just on winxp. Tomorrow will try on mac.

Comment 7

[Honza Bambas (:mayhemer)]

On mac confirmed too.

Comment 8

[Ryan VanderMeulen [:RyanVM]]

I can confirm that this patch fixed it for me.

Comment 9

[Serge Gautherie (:sgautherie)]

Comment on attachment 339380 [details] [diff] [review]
fix v1
[Backout: Comment 10]

http://hg.mozilla.org/mozilla-central/rev/9841b5e5cf10

Comment 10

[Serge Gautherie (:sgautherie)]

Comment on attachment 339380 [details] [diff] [review]
fix v1
[Backout: Comment 10]

This caused
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1222230434.1222232908.30952.gz
Win2k3 comm-central dep unit test on 2008/09/23 21:27:14
{
Traceback (most recent call last):
  File "runtests.py", line 17, in <module>
    import automation
  File "d:\builds\slave\comm-central-win32\build\objdir\mozilla\_tests\testing\mochitest\automation.py", line 43
    IS_TEST_BUILD = 
                   ^
SyntaxError: invalid syntax
}

Backout:
http://hg.mozilla.org/mozilla-central/rev/6e6ac7a9540d
http://hg.mozilla.org/mozilla-central/rev/7c8d3d111429

Comment 11

[Serge Gautherie (:sgautherie)]

(In reply to comment #10)
> http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1222230434.1222232908.30952.gz
> Win2k3 comm-central dep unit test on 2008/09/23 21:27:14

and
http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1222231212.1222234157.1273.gz
Linux comm-central dep unit test on 2008/09/23 21:40:12

Comment 12

[Serge Gautherie (:sgautherie)]

and
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1222232271.1222234298.1629.gz
WINNT 5.2 mozilla-central qm-win2k3-moz2-01 dep unit test on 2008/09/23 21:57:51
and
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1222233420.1222235296.4214.gz
WINNT 5.2 mozilla-central qm-win2k3-03 dep unit test on 2008/09/23 22:17:00

Not comm-central specific then.

Comment 13

[Honza Bambas (:mayhemer)]

I remember we have already had this problem in the past. build/pgo/automation.py.in must be touched in all trees, because tinderboxes don't perform a full build and just changing Makefiles that re-generates automation.py is not enough.

I will add some dummy change to the automation.py.in file and let's check in again.

Sorry, I completely forgot about this issue.

Comment 14

[Honza Bambas (:mayhemer)]

Attachment: fix v2 [Checkin: Comment 15]

Oops, mistake is elsewhere. I messed up name of the variable in mochitest/Makefile.in. This is fixed and tested patch.

Comment 15

[Serge Gautherie (:sgautherie)]

Comment on attachment 340111 [details] [diff] [review]
fix v2
[Checkin: Comment 15]

http://hg.mozilla.org/mozilla-central/rev/48f9681b965c