481865 - computing 1^b mod 2^32 maybe related to lcms crash

Status: RESOLVED FIXED

(Core :: Graphics: Color Management, defect)

Description

[georgi - hopefully not receiving bugspam]

Attachment: icc4.jpg - crashes

Breakpoint 1, ReadLUT8 (Icc=0x7f3a38289000, NewLUT=0x7f3a382bb000, 
    sig=icSigAToB0Tag)
    at /opt/pub/firefox-central/src/modules/lcms/src/cmsio1.c:568
568            nTabSize = (NewLUT -> OutputChan * uipow(NewLUT->cLutPoints,

(gdb) p NewLUT->cLutPoints
$1 = 1
(gdb) p uipow(NewLUT->cLutPoints,NewLUT->InputChan)
$2 = 1
(gdb) next
571            if (nTabSize > 0) {
(gdb) p nTabSize
$3 = 1
(gdb) p NewLUT -> OutputChan
$4 = 1
(gdb) cont
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007f3a326ddd94 in cmsEvalLUT (Lut=0x7f3a382bb000, In=0x7fff4814dc90, 
    Out=0x7fff4814dcd0)
    at /opt/pub/firefox-central/src/modules/lcms/src/cmslut.c:478
#2  0x00007f3a326e70c2 in LUTtoPCS (p=0x7f3a26cb3880, In=0x7fff4814dc90, 
    Out=0x7fff4814dcd0)
    at /opt/pub/firefox-central/src/modules/lcms/src/cmsxform.c:199
#3  0x00007f3a326e7326 in NormalXFORM (p=0x7f3a26cb3880, in=0x7fff4814de20, 
    out=0x7fff4814de00, Size=1)
    at /opt/pub/firefox-central/src/modules/lcms/src/cmsxform.c:312
#4  0x00007f3a326ea88f in cmsDoTransform (Transform=0x7f3a26cb3880, 
    InputBuffer=0x7fff4814de20, OutputBuffer=0x7fff4814de00, Size=1)
    at /opt/pub/firefox-central/src/modules/lcms/src/cmsxform.c:1989
#5  0x00007f3a326d2966 in _cmsComputePrelinearizationTablesFromXFORM (
    h=0x7fff4814deb0, nTransforms=1, Grid=0x7f3a382be000)
    at /opt/pub/firefox-central/src/modules/lcms/src/cmsgmt.c:1121
#6  0x00007f3a326eb9ac in _cmsPrecalculateDeviceLink (h=0x7f3a26cb3880, 
    dwFlags=128) at /opt/pub/firefox-central/src/modules/lcms/src/cmssamp.c:271
#7  0x00007f3a326ea4dc in cmsCreateProofingTransform (

Comment 1

[georgi - hopefully not receiving bugspam]

a^0 seems similar:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007f626a89fd94 in cmsEvalLUT (Lut=0x7f625fddf000, In=0x7fff81adb630, 
    Out=0x7fff81adb670)
(gdb) frame 1
#1  0x00007f626a89fd94 in cmsEvalLUT (Lut=0x7f625fddf000, In=0x7fff81adb630, 
    Out=0x7fff81adb670)
    at /opt/pub/firefox-central/src/modules/lcms/src/cmslut.c:478
478                 Lut ->CLut16params.Interp3D(StageABC, StageLMN, Lut -> T, &Lut -> CLut16params);
(gdb) p Lut
$5 = (LPLUT) 0x7f625fddf000
(gdb) p Lut ->CLut16params.Interp3D
$6 = (_cms3DLERP) 0
(gdb)

Comment 2

[georgi - hopefully not receiving bugspam]

the overflowing part at the breakpoint is in Bug 481378 Comment #12

Comment 3

[Jeff Muizelaar [:jrmuizel]]

Fixed in trunk with:
http://hg.mozilla.org/mozilla-central/rev/eae0178c3e15