1372686 - BuddyBuild - 90% of Test Failure/Crash when running XCUI/UITests

Status: RESOLVED FIXED

(Firefox for iOS :: Build & Test, defect)

Description

[ugarlapati]

BB - Test Failure:

When running XCUITest cases or UITests cases(Internally all tests are passing but coming to BB they are falling).

> Open Settings then check compacts tabs element Use Compact Tabs and swipe up,then swipe up till viable to help element,then tap help element.This flow test is working fine in local xcode.But in BB it is failing.

LOGS:https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/59401dfaa025be0001fb9543#logs

Comment 1

[No-Jun Park [:njpark]]

It looks like majority of tests are failing with App crash. 
StackTrace: https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/59401dfaa025be0001fb9543/test/77878bb6-641f-47ea-8293-fa8846ebaeef/5ee7687d-b239-43d0-93d9-eed5b1c8e5c1#tests

Before the crash, there are also these message in the log:

assertion failed: file Client/Frontend/Browser/TabManager.swift, line 131

[Client] [Graphics] UIColor created with component values far outside the expected range. Set a breakpoint on UIColorBreakForOutOfRangeColorComponents to debug. This message will only be logged once.

Last time it ran successfully was 6 days ago:
https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/5938a195d95e1200012013d7

Comment 2

[No-Jun Park [:njpark]]

Strangely, as mentioned in Comment 1, this only happens in Buddybuild, is it possible that somehow BB has stale code?  (all branches are up to date) The test WAS failing locally yesterday, but after Farhan's revert fix, it is no longer reproducible locally. Farhan, any ideas?

Comment 3

[No-Jun Park [:njpark]]

So the first commit where I can see the app crash in Buddybuild is this commit:
https://github.com/mozilla-mobile/firefox-ios/commit/3df4fe83288063147c5ee4b92c7fc448cfa53cbf

Run result in BB:
https://dashboard.buddybuild.com/apps/57bf25c0f096bc01001e21e0/build/59415748b4977600018473fd

bkmunar, is it something that can be fixed quickly?  (It's probably related to device registration, so maybe that's why it's not failing locally?) Currently this commit is causing most of UI tests to fail with app crash on BB.

Comment 4

[No-Jun Park [:njpark]]

Addtional tests have confirmed that it's caused by calling inPrivateMode()

Comment 5

[:sdaswani]

Looks like our private browsing check needs to run on the main thread: https://github.com/mozilla-mobile/firefox-ios/blob/0a9f77dc0d9049e66d6668a28026bf54db5a07e4/Client/Frontend/Browser/TabManager.swift#L131

Comment 6

[:sdaswani]

Attachment: PR

Comment 7

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

Landed on master, uplifted to v8.x