1845093 - Incorrect stacktrace column number due to unicode symbols (+ differences in Firefox Devtools "Debugger" and stacktrace column calculation)

Status: RESOLVED DUPLICATE of bug 1746374

(Core :: JavaScript Engine, defect)

Description

[u695417]

Attachment: Code to reproduce the issue (can be pasted into "Console")

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/115.0

Steps to reproduce:

There is a difference how many columns "🇩🇪" and "🇩" (and other flags / regional indicators) are considered to be in Chrome, Firefox Stacktrace and Firefox Devtools.

Here's a very short example about the reported column number of new Error:

const badSymbolFlag = "🇩🇪"; console.log(new Error().stack);

In the Firefox Devtools, the new Error starts in column number 43 (same as Chrome which also considers it to be column 43, but not consistent with the Firefox stacktrace which reports column 41).

---

Background:

We use Sentry.io to group bug reports, however, most of our Firefox reports seemed bogus and we suspected a wrong stacktrace or bad source-maps.
Everything was working fine for Chrome reports.

Our JS files are generated from Webpack, and all code is usually on the same line (which has hundreds of thousands of columns).
Hence, the column location is critical for using the source-map correctly.

We have a database of country flags near the beginning of our bundle so columns are offset by 500 in most stacktrace locations.

There might be another issue (as some lines are only 498 lines off, not 500), but I'll create a separate issue when I find that out (it might just be another flag elsewhere in the code).

I'm not sure if other emojis / symbols also cause these bad stacktraces.

Actual results:

The Firefox JS stacktrace column numbers from new Error().stack do not match what Google Chrome generates (and the source-map, in the current form, works fine in Chrome). Furthermore, the Firefox Devtools "Debugger" panel does agree with the Chrome column numbers.

Expected results:

Expected consistent stacktraces (at least column and line number) in Firefox and Chrome.
Chrome seem to report correct columns.

Also expecting consistency within Firefox stack trace and the JS editor.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'DevTools::Debugger' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[u695417]

Some good information about the source-map stack was added here https://github.com/terser/terser/issues/1415#issuecomment-1649484978
Also see https://github.com/tc39/source-map-rfc/issues/5

Comment 3

[u695417]

I experimented with it myself in the hopes of fixing it, but didn't get it production-ready.
I probably don't have the time to continue working on this either (but still very interested in getting this fixed!).

However, I think I have worked around the issue by using:

    Maybe<char32_t> cp = DecodeOneUtf8CodePoint(lead, &ptr, end);
    if (cp && IsSupplementary(*cp)) {
      count++;
    }

in size_t js::unicode::CountCodePoints(const Utf8Unit* begin, const Utf8Unit* end); https://searchfox.org/mozilla-central/source/js/src/util/Text.cpp#436 (rev 3c7b40d1; not sure how to permalink).

However, I think that function is meant to count UTF8 codepoints, so this is still bad (as it now counts UTF16 code units).

I also created this test:

/* -*- indent-tabs-mode: nil; js-indent-level: 2 -*- */
/* This Source Code Form is subject to the terms of the Mozilla Public
 * License, v. 2.0. If a copy of the MPL was not distributed with this
 * file, You can obtain one at http://mozilla.org/MPL/2.0/. */

//-----------------------------------------------------------------------------
var BUGNUMBER = 1845093;
var summary = 'bad stacktrace for emoji';
var actual = '';
var expect = '';


//-----------------------------------------------------------------------------
test();
//-----------------------------------------------------------------------------

function test()
{
  printBugNumber(BUGNUMBER);
  printStatus (summary);
 
  expect = 43;

  '🇩🇪'; const e1 = new Error().stack;
  '🇩'; const e2 = new Error().stack;

  function checkError(e, expectCol) {
    
    const lines = e.split('\n');
    const cols = lines[0].split(':');
    const col = Number(cols[cols.length-1]);
    console.log(col);

    expect = expectCol;
    actual = col;

    reportCompare(expect, actual, summary);
  }

  checkError(e1, 22)
  checkError(e2, 20)
}

I then did ./mach build && ./mach --verbose jstests regress-1845093 | tee /tmp/test.log and it looked okay.

Comment 4

[u695417]

Re-reading some other issues (namely the already mentioned https://github.com/tc39/source-map-rfc/issues/5 ), this is a duplicate of https://bugzilla.mozilla.org/show_bug.cgi?id=1746374