Closed Bug 2078698 Opened 1 month ago Closed 1 month ago

Wrong regexp result: end-anchored /u character class matching an astral code point via escape fails to match (IS_CERTAINLY_ONE_CODE_POINT set for Latin1 patterns)

Categories

(Core :: JavaScript Engine, defect, P3)

defect

Tracking

()

RESOLVED FIXED
160 Branch
Tracking Status
firefox160 --- fixed

People

(Reporter: bugmon, Assigned: iain)

References

(Blocks 1 open bug)

Details

(Keywords: testcase)

Attachments

(3 files, 1 external file)

ParserImpl::ParseCharacterClass sets ClassRanges::IS_CERTAINLY_ONE_CODE_POINT on every non-negated class parsed from a one-byte (Latin1) pattern, on the assumption that such a pattern cannot contain surrogate pairs. In Unicode mode a Latin1 pattern can still put astral code points into a class through \u{...}, a 😀 escape pair, or \p{...}, so ClassRanges::max_match() returns 1 for a class that matches two code units. Assemble() passes that under-reported length to SetCurrentPositionFromEnd for end-anchored regexps, so matching starts on the trail surrogate and the only valid start position is never tried; exec returns null where a match is required.

A web page author or any script supplying a regexp can obtain a spec-incorrect non-match by using a /u regexp written in Latin1 characters whose non-negated character class matches an astral code point and which is end-anchored but not start-anchored or sticky. The demonstrated effect is a silently missed match (exec/test/match/replace); the unanchored form of the same regexp matches correctly. No memory corruption, assertion failure or sanitizer report was observed, and the under-reported length only moves the match start position forward within the subject. The d8 binary used as the reference shares the defect when its pattern string is one-byte, so the divergence relies on the testcase building the pattern as a substring of a two-byte string.

Build Info

Affected Code

File: js/src/irregexp/imported/regexp-parser.cc, line 3430-3433

      if (sizeof(CharT) == 1 && !is_negated) {
        // No surrogate pairs.
        character_class_flags |= ClassRanges::IS_CERTAINLY_ONE_CODE_POINT;
      }

File: js/src/irregexp/imported/regexp-ast.h, line 378-383

  int max_match() override {
    if (is_certainly_one_code_point()) {
      return 1;
    }
    return 2;
  }

File: js/src/irregexp/RegExpAPI.cpp, line 561-568

  bool is_start_anchored = data->tree->IsCertainlyAnchoredAtStart(budget);
  bool is_end_anchored = data->tree->IsCertainlyAnchoredAtEnd(budget);
  int max_length = data->tree->max_match();
  static const int kMaxBacksearchLimit = 1024;
  if (is_end_anchored && !is_start_anchored && !re->sticky() &&
      max_length < kMaxBacksearchLimit) {
    masm->SetCurrentPositionFromEnd(max_length);
  }

File: js/src/irregexp/imported/regexp-interpreter.cc, line 971-978

    BYTECODE(SetCurrentPositionFromEnd, by) {
      ADVANCE();
      if (subject.length() - current > by) {
        SET_CURRENT_POSITION(subject.length() - by);
        current_char = subject[current - 1];
      }
      DISPATCH();
    }

File: js/src/irregexp/imported/regexp-parser.cc, line 3677-3687

  if (content.IsOneByte()) {
    base::Vector<const uint8_t> v = content.ToOneByteVector();
    return ParserImpl<uint8_t, ParseMode::kBuildAST>{
        v.begin(), v.length(), flags, stack_limit, zone, no_gc}
        .Parse(result);
  } else {
    base::Vector<const base::uc16> v = content.ToUC16Vector();
    return ParserImpl<base::uc16, ParseMode::kBuildAST>{
        v.begin(), v.length(), flags, stack_limit, zone, no_gc}
        .Parse(result);
  }

The flag is set purely on the pattern's character width (regexp-parser.cc:3430-3433). ClassRanges::max_match() then reports 1 (regexp-ast.h:378-383), and Assemble() uses the tree's max_match() as the back-search distance for end-anchored regexps (RegExpAPI.cpp:563-568). Both back ends honour it: the bytecode interpreter (regexp-interpreter.cc:971-978) and the native macro assembler (RegExpNativeMacroAssembler.cpp:1008-1019), which matches the identical wrong result I observed with and without --no-native-regexp. The parser's character width is chosen from the pattern string's representation (regexp-parser.cc:3677-3687), which in SpiderMonkey is the RegExp source atom (RegExpAPI.cpp:770-778). The only other consumers of max_match() outside the AST are the == 0 test at regexp-parser.cc:3647 and the > 0 tests in IsCertainlyAnchoredAtStart/End (regexp-ast.cc:97,112), which an under-report of 1 instead of 2 does not change.

Exploit Chain

  1. Script constructs a /u regexp whose source text is Latin1-only, containing a non-negated character class that matches an astral code point through an escape (e.g. [\u{1F600}]) followed by an end anchor, with no start anchor and no sticky flag.
  2. The pattern is parsed by ParserImpl<uint8_t>; ParseCharacterClass sets IS_CERTAINLY_ONE_CODE_POINT, so the tree's max_match() is 1 although the class matches two code units.
  3. Assemble() sees an end-anchored, non-start-anchored, non-sticky regexp with max_length 1 and emits SetCurrentPositionFromEnd(1).
  4. On exec against a subject ending in U+1F600, matching begins one code unit before the end (the trail surrogate); the lead-surrogate position two units before the end is never tried.
  5. exec returns null (and test/match/replace behave accordingly) although the specification requires a match at index 3 of length 2, which the same engine returns for the unanchored form.

Steps to Reproduce

  1. Use an unmodified tree; build the SpiderMonkey debug shell with /app/mozconfigs/mozconfig.js.linux.debug (build_dir /firefox/obj-js-debug). No patches, prefs or extra flags.
  2. Run testcase.js through js_shell_differential_evaluator with shell_a=/firefox/obj-js-debug/dist/bin/js (engine spidermonkey) and shell_b=/app/tools/d8/d8 (engine v8): differs=true, SpiderMonkey exits 3 with 'Error: end-anchored class failed to match the final code point' at line 8, d8 exits 0.
  3. Repeat with flags_a=["--differential-testing"]: differs=true with the same error on the SpiderMonkey side.
  4. Optional direct confirmation in the SpiderMonkey shell alone: new RegExp("[\u{1F600}]$", "u").exec("abc\u{1F600}") returns null, while new RegExp("[\u{1F600}]", "u") returns a match at index 3 of length 2; the result is the same with --no-native-regexp.

Security Impact

  • Severity: Low
  • Attacker capability: Cause a spec-incorrect non-match for a specific shape of /u regular expression (Latin1 pattern text, non-negated class matching an astral code point via escape or property escape, end-anchored, not start-anchored, not sticky). Only wrong results (missed matches) were demonstrated; no memory corruption, assertion failure or sanitizer report was observed in the debug shell.
  • Preconditions: The affected regexp shape must be in use and applied to a subject ending in an astral code point matched by the class. No prefs, flags or patches are required. The cross-engine divergence additionally depends on the pattern string being a substring of a two-byte string, because d8 returns the same wrong result when its pattern string is one-byte (verified: differs=false for the plain literal form).

ASAN Report

No sanitizer report; this is a js_differential finding. Divergence reports from my own runs on the clean rebuilt debug shell:

Run 1 (no extra flags): differs=true
  shell_a argv: /firefox/obj-js-debug/dist/bin/js --fuzzing-safe /tmp/fx_audit_jsdiff_k3rnfu6h/testcase.js -> exit_code 3, succeeded=false
  stderr: /tmp/fx_audit_jsdiff_k3rnfu6h/testcase.js:8:23 Error: end-anchored class failed to match the final code point
          Stack:
            @/tmp/fx_audit_jsdiff_k3rnfu6h/testcase.js:8:23
  shell_b argv: /app/tools/d8/d8 /tmp/fx_audit_jsdiff_k3rnfu6h/testcase.js -> exit_code 0, succeeded=true, stdout/stderr empty

Run 2 (--differential-testing on the SpiderMonkey side): differs=true
  shell_a argv: /firefox/obj-js-debug/dist/bin/js --fuzzing-safe --differential-testing /tmp/fx_audit_jsdiff_n02gd4of/testcase.js -> exit_code 3, succeeded=false
  stderr: /tmp/fx_audit_jsdiff_n02gd4of/testcase.js:8:23 Error: end-anchored class failed to match the final code point
          Stack:
            @/tmp/fx_audit_jsdiff_n02gd4of/testcase.js:8:1
  shell_b argv: /app/tools/d8/d8 /tmp/fx_audit_jsdiff_n02gd4of/testcase.js -> exit_code 0, succeeded=true, stdout/stderr empty

Control variants in the SpiderMonkey debug shell (subject "abc\u{1F600}", default flags; first, second, fifth, seventh and last lines identical under --no-native-regexp):
  new RegExp("[\\u{1F600}]$", "u")            -> null
  new RegExp("[\\u{1F600}]", "u")             -> index 3 len 2
  new RegExp("^abc[\\u{1F600}]$", "u")        -> index 0 len 5
  same with "uy", lastIndex=3                 -> index 3 len 2
  new RegExp("[" + "\u{1F600}" + "]$", "u")   -> index 3 len 2 (two-byte pattern text)
  new RegExp("[\\uD83D\\uDE00]$", "u")        -> null
  new RegExp("[\\p{Emoji_Presentation}]$", "u") -> null
  new RegExp("[^a]$", "u")                    -> index 3 len 2 (negated class)
  new RegExp("\\u{1F600}$", "u")              -> index 3 len 2 (atom, no class)
  /[\u{1F600}]$/u                             -> null

V8 caveat check: plain one-byte pattern new RegExp("(?:[\\u{1F600}])$", "u") -> differs=false (SpiderMonkey exit 3, d8 exit 1).
Attached patch fix.patch — — Splinter Review
Attached file crash_stack.txt —
Attached file testcase.js —

This bug was filed automatically by an AI-powered security testing tool. The analysis, testcase, and any suggested patches were generated by an AI model and have NOT been verified by a human.

Please review all details carefully before triaging. The issue was confirmed by an independent verifier, but for non-ASAN issues, the verifier has been made less strict. Additionally, the root cause analysis and suggested fix may contain errors.

Group: core-security → javascript-core-security
Duplicate of this bug: 2078722
Blocks: sm-security
Severity: -- → S3
Priority: -- → P3

Just a correctness issue.

Group: javascript-core-security
Keywords: sec-other

This is fixed by the irregexp update in bug 2083707.

Assignee: nobody → iireland
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Status: ASSIGNED → RESOLVED
Closed: 1 month ago
Resolution: --- → FIXED
Target Milestone: --- → 160 Branch
QA Whiteboard: [qa-triage-done-c161/b160]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: