813989 - PGO Windows builds output "PGOCVT : fatal error PG0001: An unexpected internal error was detected in source file 'f:\dd\vctools\compiler\utc\src\tools\pogo\cvtpgd\cvtpgd.cpp', line 800." during the run, but still succeed

Status: RESOLVED INCOMPLETE

(Firefox Build System :: General, defect)

Description

[Ed Morley [:emorley]]

eg:
{
Generating code
e:\builds\moz2_slave\m-cen-w32-pgo\build\layout\style\nscssparser.cpp(1071) : warning C4748: /GS can not protect parameters and local variables from local buffer overrun because optimizations are disabled in function
Finished generating code
PGOCVT : fatal error PG0001: An unexpected internal error was detected in source file 'f:\dd\vctools\compiler\utc\src\tools\pogo\cvtpgd\cvtpgd.cpp', line 800.
PGOCVT : fatal error PG0001: An unexpected internal error was detected in source file 'f:\dd\vctools\compiler\utc\src\tools\pogo\cvtpgd\cvtpgd.cpp', line 858.
}

https://tbpl.mozilla.org/php/getParsedLog.php?id=17232269&tree=Firefox

We should either fix this error or if it is expected, then suppress it.

It unfortunately shows in the TBPL annotated summary due to "fatal error", so is confusing people (including myself initially) as to real cause when things break.

Comment 1

[(not currently active) Ted Mielczarek]

This is an internal compiler error, so the only thing we can do is work around it. From the context, it's possible it is occurring while compiling nscssparser.cpp. Do you have any idea of the regression range here?

Comment 2

[Ed Morley [:emorley]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #1)
> This is an internal compiler error, so the only thing we can do is work
> around it. From the context, it's possible it is occurring while compiling
> nscssparser.cpp. Do you have any idea of the regression range here?

Err... it's present on beta but not release... :-/

Comment 3

[Ed Morley [:emorley]]

Ok, bisected by inspecting the nightly build logs (since non-nightly logs get purged after 30 days).

Good:
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/10/2012-10-05-03-06-09-mozilla-central/mozilla-central-win32-nightly-bm25-build1-build89.txt.gz

Bad:
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2012/10/2012-10-06-03-05-34-mozilla-central/mozilla-central-win32-nightly-bm13-build1-build42.txt.gz

Range:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=fd724f194a1f&tochange=2da1f2bde40e

Comment 4

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Potentially relevant: bug 723127

Comment 5

[Ed Morley [:emorley]]

I'll bisect this on Try

Comment 6

[Ed Morley [:emorley]]

Turnaround time so slow wth WINNT pgo, that I'm doing several at once.

Old-to-new, built from:

https://hg.mozilla.org/mozilla-central/rev/a898a428639f
-> https://tbpl.mozilla.org/?tree=Try&rev=32b2591c814a

https://hg.mozilla.org/mozilla-central/rev/f224c1768111
-> https://tbpl.mozilla.org/?tree=Try&rev=714d0f4131d4

https://hg.mozilla.org/mozilla-central/rev/5b5981c2106b
-> https://tbpl.mozilla.org/?tree=Try&rev=e0474908fc31

https://hg.mozilla.org/mozilla-central/rev/f4c2a6f2b838
-> https://tbpl.mozilla.org/?tree=Try&rev=2886d4b224d2

Comment 7

[Ed Morley [:emorley]]

Good:
https://hg.mozilla.org/mozilla-central/rev/a898a428639f

Bad:
https://hg.mozilla.org/mozilla-central/rev/f224c1768111

Helpfully, hgweb can't display the pushloghtml of that.

Built from https://hg.mozilla.org/mozilla-central/rev/0ea0cfc7f6f4
-> https://tbpl.mozilla.org/?tree=Try&rev=fe0fb33147a4

Comment 8

[(not currently active) Ted Mielczarek]

Maybe they came into m-c in the same push? Try checking the inbound pushlog?

Comment 9

[Ed Morley [:emorley]]

Yeah they did, it's just annoying to not be able to link to a regression range if it's inside a single push.

Comment 10

[Ed Morley [:emorley]]

(In reply to Ed Morley [UTC+0; email:edmorley@moco] from comment #7)
> Built from https://hg.mozilla.org/mozilla-central/rev/0ea0cfc7f6f4
> -> https://tbpl.mozilla.org/?tree=Try&rev=fe0fb33147a4

Bad

Comment 11

[Ed Morley [:emorley]]

Built from https://hg.mozilla.org/mozilla-central/rev/a2630fb2dbfa
-> https://tbpl.mozilla.org/?tree=Try&rev=acc57cf677ac

Built from https://hg.mozilla.org/mozilla-central/rev/7dda68e6ffba
-> https://tbpl.mozilla.org/?tree=Try&rev=97295af352b0

Comment 12

[Ed Morley [:emorley]]

(In reply to Ed Morley [UTC+0; email:edmorley@moco] from comment #11)
> Built from https://hg.mozilla.org/mozilla-central/rev/a2630fb2dbfa
> -> https://tbpl.mozilla.org/?tree=Try&rev=acc57cf677ac

Good

> Built from https://hg.mozilla.org/mozilla-central/rev/7dda68e6ffba
> -> https://tbpl.mozilla.org/?tree=Try&rev=97295af352b0

Bad

New range:

Last known good: a2630fb2dbfa
First known bad: 7dda68e6ffba

In that range:
	7dda68e6ffba	Randell Jesup — Bug 797671: cleanup from importing webrtc.org update r=ted,glandium (Part is bug 778801 r=derf)
	174d772e83f4	Randell Jesup — Bug 797671: Import Webrtc.org code from stable branch 3.12 (rev 2820) rs=jesup
	6a76b682c628	Benoit Jacob — Bug 797526 - some assertions in nsDOMClassInfo should be fatal in debug builds - r=bz
	68dd2bcc51ee	Jeff Muizelaar — Bug 792199. Only decode visible images immediately when switching to a tab. r=joe This creates a new method 'StartDecoding' which does a RequestDecode and some decoding of the image.

Comment 13

[Ed Morley [:emorley]]

Presuming it's the webrtc import, one last run to confirm.

Built from https://hg.mozilla.org/mozilla-central/rev/6a76b682c628
-> https://tbpl.mozilla.org/?tree=Try&rev=92abb61cfdde

Comment 14

[Ed Morley [:emorley]]

New range:

Last known good: 6a76b682c628
First known bad: 7dda68e6ffba

In that range:
	7dda68e6ffba	Randell Jesup — Bug 797671: cleanup from importing webrtc.org update r=ted,glandium (Part is bug 778801 r=derf)
	174d772e83f4	Randell Jesup — Bug 797671: Import Webrtc.org code from stable branch 3.12 (rev 2820) rs=jesup

Comment 15

[Ed Morley [:emorley]]

Randell, are you ok to run point on this? :-)

Comment 16

[Randell Jesup [:jesup] (needinfo me)]

Sure.  Amazing it went that long without being noticed.

Almost certainly the https://hg.mozilla.org/mozilla-central/rev/174d772e83f4 patch - which is a slim 15 MB

Taking point, but wouldn't mind some help isolating it....

Comment 17

[Reuben Morais [:reuben]]

WINNT 5.2 mozilla-b2g18 build on 2013-03-20 15:27:46 PDT for push e7996a13db8b
slave: w64-ix-slave16

https://tbpl.mozilla.org/php/getParsedLog.php?id=20910450&tree=Mozilla-B2g18