Closed Bug 1915813 Opened 2 years ago Closed 1 year ago

Crash in [@ qcms::transform_avx::qcms_transform_data_template_lut_avx]

Categories

(Core :: Graphics: Color Management, defect)

defect

Tracking

()

RESOLVED FIXED
132 Branch
Tracking Status
firefox-esr115 131+ fixed
firefox-esr128 131+ fixed
firefox130 - wontfix
firefox131 + fixed
firefox132 + fixed

People

(Reporter: yannis, Assigned: yannis)

References

(Blocks 1 open bug)

Details

(Keywords: csectype-bounds, sec-vector, Whiteboard: [adv-main131-][adv-esr128.3-][adv-esr115.16-])

Crash Data

Attachments

(6 files, 1 obsolete file)

Example crash: here

Top frames in call stack:

00 xul!qcms::transform_avx::qcms_transform_data_template_lut_avx+0x24f [/builds/worker/checkouts/gecko/gfx/qcms/src/transform_avx.rs @ 142]
01 xul!qcms::transform_avx::qcms_transform_data_rgba_out_lut_avx+0x27d [/builds/worker/checkouts/gecko/gfx/qcms/src/transform_avx.rs @ 217]
02 xul!mozilla::image::ColorManagementFilter<mozilla::image::SwizzleFilter<mozilla::image::DownscalingFilter<mozilla::image::SurfaceSink> > >::DoAdvanceRowFromBuffer+0x21 [/builds/worker/checkouts/gecko/image/SurfaceFilters.h @ 164]
03 xul!mozilla::image::SurfaceFilter::AdvanceRow+0x17 [/builds/worker/checkouts/gecko/image/SurfacePipe.h @ 141]
04 xul!mozilla::image::SurfaceFilter::WriteBuffer+0x23 [/builds/worker/checkouts/gecko/image/SurfacePipe.h @ 300]
05 xul!mozilla::image::SurfacePipe::WriteBuffer+0x2a [/builds/worker/checkouts/gecko/image/SurfacePipe.h @ 705]
06 xul!mozilla::image::nsPNGDecoder::WriteRow+0x59 [/builds/worker/checkouts/gecko/image/decoders/nsPNGDecoder.cpp @ 853]
07 xul!MOZ_PNG_push_have_row+0x6 [/builds/worker/checkouts/gecko/media/libpng/pngpread.c @ 1217]
08 xul!MOZ_PNG_push_proc_row+0x1c2 [/builds/worker/checkouts/gecko/media/libpng/pngpread.c @ 0]
09 xul!MOZ_PNG_proc_IDAT_data+0x2e1 [/builds/worker/checkouts/gecko/media/libpng/pngpread.c @ 887]
0a xul!MOZ_PNG_push_read_IDAT+0x7a4 [/builds/worker/checkouts/gecko/media/libpng/pngpread.c @ 763]
0b xul!MOZ_PNG_proc_some_data+0xbef [/builds/worker/checkouts/gecko/media/libpng/pngpread.c @ 115]
0c xul!MOZ_PNG_process_data+0xc6e [/builds/worker/checkouts/gecko/media/libpng/pngpread.c @ 46]
0d xul!mozilla::image::nsPNGDecoder::ReadPNGData+0xd70 [/builds/worker/checkouts/gecko/image/decoders/nsPNGDecoder.cpp @ 394]

Somehow I can reproduce this crash very reliably in Nightly and Release as follows (that's how I discovered it) [edit: see better STR in comment 1]:

  • Install TTD.exe as a standalone command line utility.
  • In Firefox, install and enable the 1password addon. I need to have logged into my account once through the addon.
  • In Firefox, navigate to about:config and set browser.launcherProcess.enabled to false. Close Firefox.
  • In an administrator command prompt, run Firefox through TTD, e.g. TTD.exe -out C:\traces "C:\Program Files\Firefox Nightly\firefox.exe".
  • Do other things while waiting for Firefox to start (it will be slow because of TTD).
  • Click the Firefox window and navigate to lemonde.fr.

The navigation step may be optional, however the 1password addon seems required. Based on crashes from other users, maybe the Grammarly and/or Rakuten addons could also help reproduce the crash.

Because by definition TTD records a time-travel debugging trace, I have such a trace available that I can share privately to help the investigation.

See Also: → 1915433

After all this does not appear to be related to addons. In my case the call stack is the following:

00 xul!qcms::transform_avx::qcms_transform_data_template_lut_avx+0x23b [/builds/worker/checkouts/gecko/gfx/qcms/src/transform_avx.rs @ 140]
01 xul!qcms::transform_avx::qcms_transform_data_bgra_out_lut_avx+0x269 [/builds/worker/checkouts/gecko/gfx/qcms/src/transform_avx.rs @ 227]
02 xul!mozilla::image::ColorManagementFilter<mozilla::image::SwizzleFilter<mozilla::image::SurfaceSink> >::DoAdvanceRowFromBuffer+0x21 [/builds/worker/checkouts/gecko/image/SurfaceFilters.h @ 164] 
03 xul!mozilla::image::SurfaceFilter::AdvanceRow+0x17 [/builds/worker/checkouts/gecko/image/SurfacePipe.h @ 141] 
04 xul!mozilla::image::SurfaceFilter::WriteBuffer+0x2a [/builds/worker/checkouts/gecko/image/SurfacePipe.h @ 300]
05 xul!mozilla::image::SurfacePipe::WriteBuffer+0x31 [/builds/worker/checkouts/gecko/image/SurfacePipe.h @ 705] 
06 xul!mozilla::image::nsBMPDecoder::FinishRow+0x31 [/builds/worker/checkouts/gecko/image/decoders/nsBMPDecoder.cpp @ 418] 
07 xul!mozilla::image::nsBMPDecoder::ReadPixelRow+0xa67 [/builds/worker/checkouts/gecko/image/decoders/nsBMPDecoder.cpp @ 1127] 
08 xul!mozilla::image::nsBMPDecoder::DoDecode::<lambda_4>::operator()+0xaa4 [/builds/worker/checkouts/gecko/image/decoders/nsBMPDecoder.cpp @ 459] 
09 xul!mozilla::image::StreamingLexer<mozilla::image::nsBMPDecoder::State,16>::BufferedRead+0x57 [/builds/worker/checkouts/gecko/image/StreamingLexer.h @ 604] 
0a xul!mozilla::image::StreamingLexer<mozilla::image::nsBMPDecoder::State,16>::Lex+0x13d [/builds/worker/checkouts/gecko/image/StreamingLexer.h @ 468] 
0b xul!mozilla::image::nsBMPDecoder::DoDecode+0x162 [/builds/worker/checkouts/gecko/image/decoders/nsBMPDecoder.cpp @ 431] 
0c xul!mozilla::image::Decoder::Decode+0x13e [/builds/worker/checkouts/gecko/image/Decoder.cpp @ 180] 
0d xul!mozilla::image::nsICODecoder::FlushContainedDecoder+0x38 [/builds/worker/checkouts/gecko/image/decoders/nsICODecoder.cpp @ 693] 
0e xul!mozilla::image::nsICODecoder::ReadBIH+0x114 [/builds/worker/checkouts/gecko/image/decoders/nsICODecoder.cpp @ 429] 
0f xul!mozilla::image::StreamingLexer<mozilla::image::ICOState,32>::Lex+0x199 [/builds/worker/checkouts/gecko/image/StreamingLexer.h @ 468] 
10 xul!mozilla::image::nsICODecoder::DoDecode+0x1c3 [/builds/worker/checkouts/gecko/image/decoders/nsICODecoder.cpp @ 647] 
11 xul!mozilla::image::Decoder::Decode+0x13e [/builds/worker/checkouts/gecko/image/Decoder.cpp @ 180] 
12 xul!mozilla::image::DecodedSurfaceProvider::Run+0x4f [/builds/worker/checkouts/gecko/image/DecodedSurfaceProvider.cpp @ 128] 
13 xul!mozilla::image::DecodePool::SyncRunIfPossible+0x2a [/builds/worker/checkouts/gecko/image/DecodePool.cpp @ 200] 
14 xul!mozilla::image::LaunchDecodingTask+0x31a [/builds/worker/checkouts/gecko/image/RasterImage.cpp @ 1137] 
15 xul!mozilla::image::RasterImage::Decode+0x3ee [/builds/worker/checkouts/gecko/image/RasterImage.cpp @ 1249] 
16 xul!mozilla::image::RasterImage::LookupFrame+0x60a [/builds/worker/checkouts/gecko/image/RasterImage.cpp @ 385] 
17 xul!mozilla::image::RasterImage::GetFrameAtSize+0x98 [/builds/worker/checkouts/gecko/image/RasterImage.cpp @ 581]
18 xul!mozilla::image::imgTools::EncodeScaledImage+0xdc [/builds/worker/checkouts/gecko/image/imgTools.cpp @ 517]
19 xul!nsFaviconService::OptimizeIconSizes+0x3a3 [/builds/worker/checkouts/gecko/toolkit/components/places/nsFaviconService.cpp @ 700] 
1a xul!nsFaviconService::SetFaviconForPage+0x4a5 [/builds/worker/checkouts/gecko/toolkit/components/places/nsFaviconService.cpp @ 326]

Thanks to the time-travel trace I saw that this occurs with the favicon for www.google.com and so here are my refined STR now:

  • Install TTD.exe as a standalone command line utility.
  • Install Firefox, navigate to google.com, make sure the favicon appears.
  • In Firefox, navigate to about:config and set browser.launcherProcess.enabled to false. Close Firefox.
  • In an administrator command prompt, run Firefox through TTD, e.g. TTD.exe -out C:\traces "C:\Program Files\Firefox Nightly\firefox.exe".
  • Once Firefox is started, navigate to google.com and wait for the favicon to appear.

Expected behavior: the favicon appears and colors are correct.

What happens: we crash (with official Nightly and Release binaries) or the colors of the favicon are yellowish (in a custom build or with ESR 115 binaries).

Attached image yellow-google.png

This is what the favicon looks like after following the STR in builds that don't crash (local builds, ESR 115).

Component: Widget → Graphics: Color Management
Attachment #9421830 - Attachment is obsolete: true
Attached file about-support.json

Adding about:support for completeness.

Group: core-security

It looks like, on my machine, this is likely caused by a poorly handled instruction in TTD's CPU emulation. Here is what the critical point looks like in my time-travel trace:

Time Travel Position: 2FE1DB:F63
xul!qcms::transform_avx::qcms_transform_data_template_lut_avx+0x4b [inlined in xul!qcms::transform_avx::qcms_transform_data_rgba_out_lut_avx+0x82]:
00007ffd`87a101f2 c4e27d1a5120    vbroadcastf128 ymm2,oword ptr [rcx+20h] ds:000002e8`823b8960=000000003f80036537882400386dd000
0:028> r xmm2
xmm2=    -1.#QNAN     -1.#QNAN  1.4013e-045 4.48416e-044
0:028> df rcx+20 L4
000002e8`823b8960    5.6698918e-005   1.6229227e-005        1.0001036                0
0:028> t
Time Travel Position: 2FE1DB:F64
xul!qcms::transform_avx::qcms_transform_data_template_lut_avx+0x51 [inlined in xul!qcms::transform_avx::qcms_transform_data_rgba_out_lut_avx+0x88]:
00007ffd`87a101f8 4983f901        cmp     r9,1
0:028> r xmm2
xmm2=    -1.#QNAN     -1.#QNAN 1.62292e-005 5.66989e-005

xmm2 holds -1.#QNAN -1.#QNAN 1.4013e-045 4.48416e-044 and then we broadcast from memory the values 5.6698918e-005 1.6229227e-005 1.0001036 0 but only the two first values get broadcasted, the third float stays at an invalid value of -1.#QNAN instead of taking the broadcasted value 1.0001036.

Here is the execution of the same instruction at the same point without TTD instrumentation:

Breakpoint 0 hit
xul!qcms::transform_avx::qcms_transform_data_template_lut_avx+0x4b [inlined in xul!qcms::transform_avx::qcms_transform_data_rgba_out_lut_avx+0x82]:
00007ffd`661c01f2 c4e27d1a5120    vbroadcastf128 ymm2,oword ptr [rcx+20h] ds:00000230`0d03d2a0=000000003f80036537882400386dd000
0:030> r xmm2
xmm2=    -1.#QNAN     -1.#QNAN  1.4013e-045 4.48416e-044
0:030> df rcx+20 L4
00000230`0d03d2a0    5.6698918e-005   1.6229227e-005        1.0001036                0
0:030> t
xul!qcms::transform_avx::qcms_transform_data_template_lut_avx+0x51 [inlined in xul!qcms::transform_avx::qcms_transform_data_rgba_out_lut_avx+0x88]:
00007ffd`661c01f8 4983f901        cmp     r9,1
0:030> r xmm2
xmm2=           0       1.0001 1.62292e-005 5.66989e-005

I'll try to make a small reproducer and email Microsoft about it. However, I highly doubt that people in the wild are relying on TTD to run into this crash, so I'm marking as security and will explain in more detail why next week (potential out-of-bounds read).

Group: core-security → gfx-core-security

For the in the wild crashes, only 12 of the 77 crashes that come up in the search do not have a non-zero bit flip confidence. So seems like mostly hardware type issues.

I have confirmed the TTD CPU emulation bug and emailed our contacts at Microsoft about it. The effect of the TTD CPU emulation bug can be somewhat replicated by replacing:

    let mat2: __m256 = _mm256_broadcast_ps(&*((*mat.offset(2isize)).as_ptr() as *const __m128));

With:

    let mut mat2 = *mat.offset(2isize);
    mat2[2] = f32::NAN;
    let mat2: __m256 = _mm256_broadcast_ps(&*(mat2.as_ptr() as *const __m128));

The NaN value in mat2[2] propagates through the various computations until it eventually reaches _mm256_cvtps_epi32:

unsafe extern "C" fn qcms_transform_data_template_lut_avx<F: Format>(
    transform: &qcms_transform,
    mut src: *const u8,
    mut dest: *mut u8,
    mut length: usize,
) {
    // ...
    while length > 3 {
        // ...
        vec_b = _mm256_mul_ps(vec_b, mat2);
        // ...
        vec_r = _mm256_add_ps(vec_r, _mm256_add_ps(vec_g, vec_b));
        vec_r = _mm256_max_ps(min, vec_r);
        vec_r = _mm256_min_ps(max, vec_r);
        result = _mm256_mul_ps(vec_r, scale);
        _mm256_store_si256(output as *mut __m256i, _mm256_cvtps_epi32(result));
        // ...
        *dest.add(F::kBIndex) = *otdata_b.offset(*output.offset(2isize) as isize);
        // ...
        *dest.add(F::kBIndex + components as usize) =
            *otdata_b.offset(*output.offset(6isize) as isize);
        // ...
    }
    // ...

Then _mm256_cvtps_epi32 converts the NaN float value to the 32-bit integer 0x80000000, so this is what output[2] and output[6] will receive. We then use this 0x80000000 value as an index of the 8192-byte array otdata_b, resulting in an out-of-bounds read.

More generally stated, the max/min/mul sequence on vec_r is present to guarantee that output only contains values between 0 and 8191, but it fails to do that if the vec_r has somehow been contaminated by NaN values. I tested locally with all non-NaN f32 values and confirmed that the issue only exists with NaN values. I think such NaN contamination can only occur through transform.matrix and transform.input_gamma_table_{r,g,b}.

In order to fix this, which would let us use current versions TTD without crashing, we could simply swap the order of arguments either in the _mm256_min_ps or in the _mm256_max_ps operation. The description for these operations (max, min) says: "If only one value is a NaN (SNaN or QNaN) for this instruction, the second operand (source operand), either a NaN or a valid floating-point value, is written to the result.". Swapping in _mm256_max_ps would map NaN to 0, while swapping in _mm256_min_ps would map NaN to 8191.

Microsoft TTD has a CPU emulation bug that makes Firefox crash when ran
under TTD. This patch mitigates the issue.

Assignee: nobody → yjuglaret
Status: NEW → ASSIGNED

This is sort of a CPU bug. Just a software CPU.

Blocks: cpu-bugs

[Tracking Requested - why for this release]: The bug is in pretty old code so all builds are affected. Not sure if Thunderbird uses that code.

Microsoft TTD has a CPU emulation bug that makes Firefox crash when ran
under TTD. This patch mitigates the issue.

Original Revision: https://phabricator.services.mozilla.com/D220829

Attachment #9422940 - Flags: approval-mozilla-beta?

Comment on attachment 9422091 [details]
Bug 1915813 - Fix a crash with Microsoft TTD. r=#gfx-reviewers

Security Approval Request

  • How easily could an exploit be constructed based on the patch?: Even though this bug involves a memory corruption it looks unlikely that it can be used to construct an exploit. First, it is unclear how to trigger the corruption without the CPU emulation bug, if at all possible. Second, the memory corruption is an OOB read at a fixed large offset, and it seems unlikely that an attacker can reliably put an object that would be interesting for them to read as part of an exploit at the proper location.
  • Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem?: No
  • Which branches (beta, release, and/or ESR) are affected by this flaw, and do the release status flags reflect this affected/unaffected state correctly?: beta, release, ESR
  • If not all supported branches, which bug introduced the flaw?: None
  • Do you have backports for the affected branches?: Yes
  • If not, how different, hard to create, and risky will they be?:
  • How likely is this patch to cause regressions; how much testing does it need?: Very unlikely to cause regressions, the patch just swaps argument order on an operation to handle NaN values differently, non-NaN values are treated the same no matter the order of arguments.
  • Is the patch ready to land after security approval is given?: Yes
  • Is Android affected?: Unknown
Attachment #9422091 - Flags: sec-approval?

I guess Android x86 could be affected by the OOB (assuming it is reachable without the CPU bug), not sure if that counts.

Comment on attachment 9422091 [details]
Bug 1915813 - Fix a crash with Microsoft TTD. r=#gfx-reviewers

Approved to land and request uplift

Attachment #9422091 - Flags: sec-approval? → sec-approval+

beta Uplift Approval Request

  • User impact if declined: Crashing when using Firefox under Microsoft TTD and potential presence of an OOB read memory corruption.
  • Code covered by automated testing: yes
  • Fix verified in Nightly: no
  • Needs manual QE test: no
  • Steps to reproduce for manual QE testing: -
  • Risk associated with taking this patch: Low
  • Explanation of risk level: The patch just changes the order of arguments on an operation whose behavior only depends on argument order for NaN values. The patch does not affect the flow of computation for non-NaN values, and NaN values are unexpected and currently cause crashes.
  • String changes made/needed: No
  • Is Android affected?: no
Pushed by yjuglaret@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/3f2eafd31fe8 Fix a crash with Microsoft TTD. r=gfx-reviewers,nical,jrmuizel
Group: gfx-core-security → core-security-release
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 132 Branch

Fix verified in the latest Nightly. The CPU emulation bug still exists, so I still get a NaN value flowing around when it should not:

Time Travel Position: 2CBB3E:1C90
xul!qcms::transform_avx::qcms_transform_data_template_lut_avx+0x4b [inlined in xul!qcms::transform_avx::qcms_transform_data_rgba_out_lut_avx+0x82]:
00007ffb`514e9812 c4e27d1a5120    vbroadcastf128 ymm2,oword ptr [rcx+20h] ds:00000223`f603e960=000000003f80036537882400386dd000
0:027> rY
ymm2=0 0 0 0 -1.#QNAN -1.#QNAN 1.4013e-045 3.36312e-044
0:027> t
Time Travel Position: 2CBB3E:1C91
0:027> rY
ymm2=0 1.0001 1.62292e-005 5.66989e-005 -1.#QNAN -1.#QNAN 1.62292e-005 5.66989e-005

And the NaN value still propagates a bit, but eventually the vmaxps operation replaces it by zero:

Time Travel Position: 2CBB3E:1CAF
xul!qcms::transform_avx::qcms_transform_data_template_lut_avx+0x1a8 [inlined in xul!qcms::transform_avx::qcms_transform_data_rgba_out_lut_avx+0x1df]:
00007ffb`514e996f c5d45feb        vmaxps  ymm5,ymm5,ymm3
0:027> rY
ymm3=0 0 0 0 0 0 0 0
ymm5=0 0 0 0 -1.#QNAN -1.#QNAN 0 0
0:027> t
Time Travel Position: 2CBB3E:1CB0
0:027> rY
ymm5=0 0 0 0 0 0 0 0

And we end up converting 0 rather than NaN for the value that we use as an offset (as described in comment 8), so we don't crash.

beta Uplift Approval Request

  • User impact if declined: Crashing when using Firefox under Microsoft TTD and potential presence of an OOB read memory corruption.
  • Code covered by automated testing: yes
  • Fix verified in Nightly: yes
  • Needs manual QE test: no
  • Steps to reproduce for manual QE testing: -
  • Risk associated with taking this patch: Low
  • Explanation of risk level: The patch just changes the order of arguments on an operation whose behavior only depends on argument order for NaN values. The patch does not affect the flow of computation for non-NaN values, and NaN values are unexpected and currently cause crashes.
  • String changes made/needed: No
  • Is Android affected?: no
Attachment #9422940 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
QA Whiteboard: [post-critsmash-triage]
Flags: qe-verify-

Microsoft TTD has a CPU emulation bug that makes Firefox crash when ran
under TTD. This patch mitigates the issue.

Original Revision: https://phabricator.services.mozilla.com/D220829

Attachment #9424359 - Flags: approval-mozilla-esr128?

Microsoft TTD has a CPU emulation bug that makes Firefox crash when ran
under TTD. This patch mitigates the issue.

Original Revision: https://phabricator.services.mozilla.com/D220829

Attachment #9424364 - Flags: approval-mozilla-esr115?

esr128 Uplift Approval Request

  • User impact if declined: Crashing when using Firefox under Microsoft TTD and potential presence of an OOB read memory corruption.
  • Code covered by automated testing: yes
  • Fix verified in Nightly: yes
  • Needs manual QE test: no
  • Steps to reproduce for manual QE testing: -
  • Risk associated with taking this patch: Low
  • Explanation of risk level: The patch just changes the order of arguments on an operation whose behavior only depends on argument order for NaN values. The patch does not affect the flow of computation for non-NaN values, and NaN values are unexpected and currently cause crashes.
  • String changes made/needed: No
  • Is Android affected?: no

esr115 Uplift Approval Request

  • User impact if declined: Crashing when using Firefox under Microsoft TTD and potential presence of an OOB read memory corruption.
  • Code covered by automated testing: yes
  • Fix verified in Nightly: yes
  • Needs manual QE test: no
  • Steps to reproduce for manual QE testing: -
  • Risk associated with taking this patch: Low
  • Explanation of risk level: The patch just changes the order of arguments on an operation whose behavior only depends on argument order for NaN values. The patch does not affect the flow of computation for non-NaN values, and NaN values are unexpected and currently cause crashes.
  • String changes made/needed: No
  • Is Android affected?: no
Attachment #9424359 - Flags: approval-mozilla-esr128? → approval-mozilla-esr128+

Comment on attachment 9424364 [details]
Bug 1915813 - Fix a crash with Microsoft TTD.

Approved for 115.16esr

Attachment #9424364 - Flags: approval-mozilla-esr115? → approval-mozilla-esr115+

Microsoft has also confirmed that the CPU emulation bug will be fixed in the next release of TTD.exe.

Whiteboard: [adv-main131-]
Whiteboard: [adv-main131-] → [adv-main131-][adv-esr128.3-]
Whiteboard: [adv-main131-][adv-esr128.3-] → [adv-main131-][adv-esr128.3-][adv-esr115.16-]
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: