BuddyBuild - 90% of Test Failure/Crash when running XCUI/UITests

RESOLVED FIXED

Status

()

Firefox for iOS
Build & Test
RESOLVED FIXED
6 months ago
6 months ago

People

(Reporter: ugarlapati, Unassigned)

Tracking

unspecified
Other
iOS

Firefox Tracking Flags

(fxios8.0+)

Details

(Whiteboard: [mma] [MobileCore])

Attachments

(1 attachment)

PR
55 bytes, text/x-github-pull-request
Susheel Daswani
: review?
st3fan
Details | Review | Splinter Review
(Reporter)

Description

6 months ago
BB - Test Failure:

When running XCUITest cases or UITests cases(Internally all tests are passing but coming to BB they are falling).

> Open Settings then check compacts tabs element Use Compact Tabs and swipe up,then swipe up till viable to help element,then tap help element.This flow test is working fine in local xcode.But in BB it is failing.

LOGS:https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/59401dfaa025be0001fb9543#logs

Updated

6 months ago
tracking-fxios: --- → ?

Comment 1

6 months ago
It looks like majority of tests are failing with App crash. 
StackTrace: https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/59401dfaa025be0001fb9543/test/77878bb6-641f-47ea-8293-fa8846ebaeef/5ee7687d-b239-43d0-93d9-eed5b1c8e5c1#tests

Before the crash, there are also these message in the log:

assertion failed: file Client/Frontend/Browser/TabManager.swift, line 131

[Client] [Graphics] UIColor created with component values far outside the expected range. Set a breakpoint on UIColorBreakForOutOfRangeColorComponents to debug. This message will only be logged once.

Last time it ran successfully was 6 days ago:
https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/5938a195d95e1200012013d7

Updated

6 months ago
Summary: BB - Test Failure/Crash when running XCUI/UITests → BuddyBuild - 90% of Test Failure/Crash when running XCUI/UITests

Comment 2

6 months ago
Strangely, as mentioned in Comment 1, this only happens in Buddybuild, is it possible that somehow BB has stale code?  (all branches are up to date) The test WAS failing locally yesterday, but after Farhan's revert fix, it is no longer reproducible locally. Farhan, any ideas?
Flags: needinfo?(fpatel)

Comment 3

6 months ago
So the first commit where I can see the app crash in Buddybuild is this commit:
https://github.com/mozilla-mobile/firefox-ios/commit/3df4fe83288063147c5ee4b92c7fc448cfa53cbf

Run result in BB:
https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/59415748b4977600018473fd

bkmunar, is it something that can be fixed quickly?  (It's probably related to device registration, so maybe that's why it's not failing locally?) Currently this commit is causing most of UI tests to fail with app crash on BB.
Flags: needinfo?(fpatel) → needinfo?(bmunar)

Comment 4

6 months ago
Addtional tests have confirmed that it's caused by calling inPrivateMode()

Comment 5

6 months ago
Looks like our private browsing check needs to run on the main thread: https://github.com/mozilla-mobile/firefox-ios/blob/0a9f77dc0d9049e66d6668a28026bf54db5a07e4/Client/Frontend/Browser/TabManager.swift#L131
Blocks: 1351446
Flags: needinfo?(bmunar)
Whiteboard: [mma] [MobileCore][needsuplift]

Comment 6

6 months ago
Created attachment 8877764 [details] [review]
PR
Attachment #8877764 - Flags: review?(sarentz)
Landed on master, uplifted to v8.x
Status: NEW → RESOLVED
Last Resolved: 6 months ago
Resolution: --- → FIXED

Updated

6 months ago
Whiteboard: [mma] [MobileCore][needsuplift] → [mma] [MobileCore]

Updated

6 months ago
tracking-fxios: ? → 8.0+
You need to log in before you can comment on or make changes to this bug.