Closed Bug 2086214 Opened 18 days ago Closed 16 days ago

Crash in [@ autofill::db::addresses::update_internal_address]

Categories

(Application Services :: General, defect)

defect

Tracking

(firefox160+ disabled, firefox161+ fixed)

RESOLVED FIXED
161 Branch
Tracking Status
firefox160 + disabled
firefox161 + fixed

People

(Reporter: clouseau-bot, Assigned: dimi)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: crash, regression)

Crash Data

Attachments

(2 external files)

Crash report: https://crash-stats.mozilla.org/report/index/1b0f8e97-bc32-4ea7-8a85-af7980261027

MOZ_CRASH Reason:

assertion `left == right` failed
  left: 0
 right: 1

Top 10 frames:

0  xul.dll  RustMozCrash(char const*, int, char const*)  mozglue/static/rust/wrappers.cpp:17
1  xul.dll  mozglue_static::panic_hook(std::panic::PanicHookInfo*)  mozglue/static/rust/lib.rs:99
2  xul.dll  core::ops::function::FnOnce::call_once<void (*)(ref$<std::panic::PanicHookInfo>)...
3  xul.dll  std::panicking::panic_with_hook()  library/std/src/panicking.rs:833
4  xul.dll  std::panicking::panic_handler::closure$0()  library/std/src/panicking.rs:698
5  xul.dll  std::sys::backtrace::__rust_end_short_backtrace<std::panicking::panic_handler::c...  library/std/src/sys/backtrace.rs:182
6  xul.dll  std::panicking::panic_handler()  library/std/src/panicking.rs:689
7  xul.dll  core::panicking::panic_fmt()  library/core/src/panicking.rs:80
8  xul.dll  core::panicking::assert_failed_inner()  library/core/src/panicking.rs:439
9  xul.dll  core::panicking::assert_failed<usize,usize>()  library/core/src/panicking.rs:394

There are 3 crashes (from 1 installation) in nightly 161 with buildid 20261027091753.

This bug looks actionable because:

  • During address sync apply, update_internal_address panics on assert_eq!(rows_changed, 1) because its UPDATE addresses_data ... WHERE guid = :guid matched 0 rows, which takes down the Firefox parent process.

    1. AddressesBridgedEngine::apply runs ConfigSyncEngine::apply, which loops over the incoming states, plans an action for each and calls apply_incoming_action (engine.rs:157).
    2. For an Update, UpdateLocalGuid or ResurrectRemoteTombstone action, apply_incoming_action calls update_local_record, which calls update_internal_address with the record's guid (incoming.rs:313).
    3. No row in addresses_data has that guid at that moment, so the UPDATE changes 0 rows.
    4. assert_eq!(rows_changed, 1) at addresses.rs:363 fails (left: 0 right: 1) and the panic aborts the parent process on BgIOThreadPool.

    The crash reason assertion left == right failed left: 0 right: 1 and frames 10–14 match this chain exactly. Inlining hides which of the three update_local_record call sites ran, and why the planned row is absent is not established. Look first at the UpdateLocalGuid path (does common_change_guid move the row before the update, or could it change no row?) and at whether planning can see a local record that is gone by the time the update runs. Also consider whether this invariant should return an error instead of panicking in the parent process.

  • The failing code comes from 4b393efa5019 (gh) (bug 2019144) by :tcampbell.

Code references:

:tcampbell, can you have a look please?

Filed automatically by Clouseau, which analyses nightly crashes with an LLM. Nothing above was written or checked by a human. Please close it as INVALID if it is wrong — that is useful feedback, not a nuisance.

This is not an independent Mozilla discovery: the crash is known here only because somebody submitted the crash report linked at the top. If this duplicates an existing report, please resolve THIS bug as the duplicate and leave the credit with the earlier reporter.

Flags: needinfo?(tcampbell)

I've been repeatedly seeing this crash on recent builds on my main computer on a very-long-running profile connected to multiple devices with sync. Happy to help if you need any information, though currently, this is stopping me from using one of the core parts of my productivity, i.e. my main browser on my main computer, so can't wait for long until I have to find some way to solve this.

Follow-up to comment 0: this isn't a regression from a changeset. It started when the Nimbus rollouts that enable the Rust address store went live:

Both target 100% of users on version >= 160 and set extensions.formautofill.addresses.storage.rust.enabled to true, through the Nimbus feature added in bug 2080704. Once the migration completes, Sync switches to the Rust bridged engine (bug 2062542), which is where this assertion fails.

The first crash came at 17:44 UTC, and there are none in the previous year. Builds that were already shipping before the rollout show no crashes with this signature until it went live, then all of them start crashing:

build parent crashes before 17:16 UTC (installs) this signature before this signature after (installs)
20261023095057 141 (103) 0 3 (1)
20261024215018 149 (95) 0 5 (2)
20261026091849 67 (47) 0 1 (1)
20261026160927 (160.0b1) 107 (75) 0 31 (4)
20261026210443 26 (22) 0 3 (3)

None of the application-services vendorings explain it. The vendored autofill files on the stack haven't changed since 2026-10-15, and the 2026-10-27 vendoring (bug 2086069) landed at 21:20 UTC, after the first crash. The assert_eq! at addresses.rs:363 is longstanding app-services code; what's new is desktop running it on migrated data.

The commit named in comment 0 (4b393efa5019) only vendored that line, so the needinfo moves from :tcampbell to :dimi. Marking this as regressed by bug 2080704, the change the rollout goes through; the crashing path is the one from bug 2062542.

:dimi, this crashes the parent process, in a loop at startup for kairo (comment 1). Could the rollouts be paused while you look at it?

Bug 2080704 also has a pending release uplift whose stated purpose is to get signals from this migration. Enrolling crashes profiles that sync addresses, so that uplift should probably wait for this bug.

Kairo, unchecking Addresses in the Sync settings (services.sync.engine.addresses = false) should stop the crash until there's a fix, since this panic only happens while the addresses engine syncs.

Flags: needinfo?(tcampbell) → needinfo?(dlee)
Regressed by: 2080704
Assignee: nobody → dlee
Status: NEW → ASSIGNED
Flags: needinfo?(dlee)

Update: both nightly and beta rollout were disabled

And just as a note, turning services.sync.engine.addresses off via user.js seemed to not work for me (I crashed right after startup so couldn't go into about:config), so I ended up editing prefs.js and turning off services.sync.engine.addresses as well as extensions.formautofill.addresses.storage.rust.enabled and extensions.formautofill.addresses.storage.rust.active and my Nightly has been stable for hours now, thanks for the pointers. Maybe doing all those three is overkill for now, but I wanted to make sure for now and when this is fixed will go back and turn them on again. Thanks for quickly reacting and giving pointers - as well as working on the fix. Very much appreciated! :)

Authored by https://github.com/DimiDL
https://github.com/mozilla/application-services/commit/50b768862de10b0d89dd6389afbda7a8a48c771e
[main] Bug 2086214 - Don't pick a local autofill record that is also being downloaded as a sync dupe (#7698)

(via GitHub webhook, authenticated as shared-secret)

Status: ASSIGNED → RESOLVED
Closed: 17 days ago
Flags: qe-verify+
Resolution: --- → FIXED

Reopening this issue since the fix requires Firefox Desktop to integrate the changes from Application Services.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Pushed by dlee@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/6fab544fb9f2 https://hg.mozilla.org/integration/autoland/rev/b0b6ff4f5182 Vendor application-services 50b76886 to fix a crash in autofill sync de-duplication. r=bdk
Status: REOPENED → RESOLVED
Closed: 17 days ago → 16 days ago
Resolution: --- → FIXED
Target Milestone: --- → 161 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: