1974993 - 9.27 - 6.08% perf_reftest_singletons thai-reflow.html / perf_reftest_singletons thai-reflow.html (Linux, Windows) regression on Tue June 24 2025

Status: RESOLVED FIXED

(Core :: Internationalization, defect)

Description

[Treeherder Bug Filer]

Perfherder has detected a talos performance regression from push 1cba4d9944a96784fec3aed56f829119c115fb6a. As author of one of the patches included in that push, we need your help to address this regression.

Please acknowledge, and begin investigating this alert within 3 business days, or the patch(es) may be backed out in accordance with our regression policy. Our guide to handling regression bugs has information about how you can proceed with this investigation.

If you have any questions or need any help with the investigation, please reach out to [e-mail]. Alternatively, you can find help on Slack by joining #perf-help, and on Matrix you can find help by joining #perftest.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
9%	perf_reftest_singletons thai-reflow.html	linux1804-64-shippable-qr	e10s fission stylo webrender	72.94 -> 79.70
6%	perf_reftest_singletons thai-reflow.html	windows11-64-24h2-shippable	e10s fission stylo webrender	30.65 -> 32.52

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask [e-mail] to do that for you.

You can run all of these tests on try with ./mach try perf --alert 45749

The following documentation link provides more information about this command.

Comment 1

[Makoto Kato [:m_kato]]

nsLineBreaker benchmark for thai (https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&series=mozilla-central,5440391,1,6&timerange=7776000) shows small perf regression.
LineBreaker::ComputeBreakPositions benchmark for thai
(https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&series=mozilla-central,5440377,1,6&timerange=7776000) might not... But other language may have a regression.
I will investigate ICU4X side.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1960300

Comment 3

[Dianna Smith [:diannaS]]

:m_kato can you set a priority/severity on this?

Comment 4

[Makoto Kato [:m_kato]]

This seems to be ICU4X side. And we have to upgrade to 2.0 for some newer features.

Comment 5

[Makoto Kato [:m_kato]]

https://github.com/unicode-org/icu4x/pull/5924 causes pref regression.

Line Break/UTF16/Th/lstm
                        time:   [335.07 µs 336.35 µs 337.94 µs]
                        change: [+16.697% +17.471% +18.280%] (p = 0.00 < 0.05)
                        Performance has regressed.

But https://github.com/unicode-org/icu4x/issues/6562 is already filed.

Comment 6

[Makoto Kato [:m_kato]]

LSTM calls ZeroVec.iter() in MatrixZero.to_own() from compute_hc.

If we revert https://github.com/unicode-org/icu4x/pull/5924 like the following, this might be fixed.

--- a/components/segmenter/src/complex/lstm/matrix.rs
+++ b/components/segmenter/src/complex/lstm/matrix.rs
@@ -416,17 +416,17 @@ impl<'a> From<&'a crate::provider::LstmMatrix3<'a>> for MatrixZero<'a, 3> {
         }
     }
 }
 
 impl<'a, const D: usize> MatrixZero<'a, D> {
     #[expect(clippy::wrong_self_convention)] // same convention as slice::to_vec
     pub(super) fn to_owned(&self) -> MatrixOwned<D> {
         MatrixOwned {
-            data: self.data.iter().collect(),
+            data: self.data.iter_unstable().collect(),
             dims: self.dims,
         }
     }
 
     pub(super) fn as_slice(&self) -> &ZeroSlice<f32> {
         self.data
     }
 
diff --git a/utils/zerovec/src/zerovec/slice.rs b/utils/zerovec/src/zerovec/slice.rs
index 59351878a5..e545a9d229 100644
--- a/utils/zerovec/src/zerovec/slice.rs
+++ b/utils/zerovec/src/zerovec/slice.rs
@@ -362,16 +362,22 @@ where
     /// assert_eq!(it.next(), Some(32973));
     /// assert_eq!(it.next(), None);
     /// ```
     #[inline]
     pub fn iter<'a>(&'a self) -> ZeroSliceIter<'a, T> {
         ZeroSliceIter(self.as_ule_slice().iter())
     }
 
+    pub fn iter_unstable(
+        &self,
+    ) -> impl DoubleEndedIterator<Item = T> + ExactSizeIterator<Item = T> + '_ {
+        self.as_ule_slice().iter().copied().map(T::from_unaligned)
+    }
+
     /// Returns a tuple with the first element and a subslice of the remaining elements.
     ///
     /// # Example
     ///
     /// ```
     /// use zerovec::ule::AsULE;
     /// use zerovec::ZeroSlice;
     ///

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

It has been over 7 days with no activity on this performance regression.

:m_kato, since you are the author of the regressor, bug 1960300, which triggered this performance alert, could you please provide a progress update?

If this regression is something that fixes a bug, changes the baseline of the regression metrics, or otherwise will not be fixed, please consider closing it as WONTFIX. See this documentation for more information on how to handle regressions.

For additional information/help, please needinfo the performance sheriff who filed this alert (they can be found in comment #0), or reach out in #perftest, or #perfsheriffs on Element.

For more information, please visit BugBot documentation.

Comment 8

[Brian Birtles (:birtles)]

https://github.com/unicode-org/icu4x/pull/6764 has been merged so I believe we are waiting to see if the ICU4X maintainers agree to create a dot release. Otherwise, we will need to create a temporary fork to apply this fix.

Kato-san, have you heard anything from the ICU4X maintainers?

Comment 9

[Makoto Kato [:m_kato]]

(In reply to Brian Birtles (:birtles) from comment #8)

https://github.com/unicode-org/icu4x/pull/6764 has been merged so I believe we are waiting to see if the ICU4X maintainers agree to create a dot release. Otherwise, we will need to create a temporary fork to apply this fix.

Kato-san, have you heard anything from the ICU4X maintainers?

Manish will release newer version by https://github.com/unicode-org/icu4x/pull/6796.

Comment 10

[Manish Goregaokar [:manishearth]]

For future reference; icu_* crates are on a relatively slow release schedule (~twice a year), though we can make patch releases uplifting small patches on request. Other "utils" crates from ICU4X can usually be released at any time by asking.

Comment 11

[Makoto Kato [:m_kato]]

Attachment: Bug 1974993 - Update zerovec to 0.11.3 r=#supply-chain-reviewers!,TYLin!

Newer version of zerovec includes a performance improvement for East
Asian language's segmenters.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/97938ea2936e
https://hg.mozilla.org/integration/autoland/rev/17db4c7a8c2b
Update zerovec to 0.11.3 r=TYLin,supply-chain-reviewers

Comment 13

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/17db4c7a8c2b

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:m_kato, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
  • See https://wiki.mozilla.org/Release_Management/Requesting_an_Uplift for documentation on how to request an uplift.
• If no, please set status-firefox142 to wontfix.

For more information, please visit BugBot documentation.

Comment 15

[Makoto Kato [:m_kato]]

Perf data: https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightCommonAlerts=0&replicates=0&series=autoland,234296,1,1&timerange=7776000

Comment 16

[Makoto Kato [:m_kato]]

Comment on attachment 9505078 [details]
Bug 1974993 - Update zerovec to 0.11.3 r=#supply-chain-reviewers!,TYLin!

Beta/Release Uplift Approval Request

• User impact if declined/Reason for urgency: After updating ICU4X to 2.0, layout for East Asian language has a performance regression
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Low, this fix updates zerovec crate that is used by data structure. Also, this update is minor release.
• String changes made/needed:
• Is Android affected?: Yes

Comment 17

[Dianna Smith [:diannaS]]

Comment on attachment 9505078 [details]
Bug 1974993 - Update zerovec to 0.11.3 r=#supply-chain-reviewers!,TYLin!

approved for 142.0b9

Comment 18

[Pulsebot]

https://github.com/mozilla-firefox/firefox/commit/d5951c689d00
https://hg.mozilla.org/releases/mozilla-beta/rev/9e8387cc9c95

Comment 19

[Acasandrei Beatrice (needinfo me)]

(In reply to Pulsebot from comment #12)

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/97938ea2936e
https://hg.mozilla.org/integration/autoland/rev/17db4c7a8c2b
Update zerovec to 0.11.3 r=TYLin,supply-chain-reviewers

Perfherder has detected a talos performance change from push 17db4c7a8c2b9f74c4075d190f46ec9d956e6a47.

If you have any questions, please reach out to a performance sheriff. Alternatively, you can find help on Slack by joining #perf-help, and on Matrix you can find help by joining #perftest.

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
8%	perf_reftest_singletons thai-reflow.html	linux1804-64-shippable-qr	e10s fission stylo webrender	79.87 -> 73.31
6%	perf_reftest_singletons thai-reflow.html	windows11-64-24h2-shippable	e10s fission stylo webrender	32.56 -> 30.72

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a performance sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 46197

The following documentation link provides more information about this command.