1880546 - firefox-android: Clarify scheduling of screenshots

Status: RESOLVED FIXED

(Release Engineering :: General, task)

Description

[Geoff Brown [:gbrown]]

"screenshots" tests, like ui-test-apk-screenshots-arm almost never run on firefox-android:
https://treeherder.mozilla.org/jobs?repo=firefox-android&searchStr=screenshot

but run frequently on oak:
https://treeherder.mozilla.org/jobs?repo=oak&searchStr=screenshot

When should they run?

Comment 1

[Geoff Brown [:gbrown]]

The screenshots cron entry was originally added in https://github.com/mozilla-mobile/fenix/commit/5b61b44f1a2f6f26541249acfd54ab70a6ba03ec, to run a screenshots-specific decision task once a week; that cron schedule is still in effect on firefox-android.

However, on firefox-android, the screenshots-D decision task rarely (never?) runs the screenshots ui-test task. I think that's because of the skip-unless optimization (the screenshots ui-test task won't be run unless the commit it runs on modified certain fenix files)...but I'm not sure that fully explains the lack of runs.

https://github.com/mozilla-mobile/firefox-android/blob/main/taskcluster/ci/ui-test-apk/kind.yml#L24
https://github.com/mozilla-mobile/firefox-android/blob/main/taskcluster/ci/ui-test-apk/kind.yml#L271

Comment 2

[Geoff Brown [:gbrown]]

Isabel, Aaron - Can either of you say how often the screenshots ui-test should run on firefox-android? It looks like we try to run it once a week (Mondays) on a cron schedule, but the actual tests almost never run.

Comment 3

[Aaron Train [:aaronmt]]

Thanks for the ping. Right, this task is setup to run weekly on Monday's. Looks like we haven't been watching this particular low-priority task recently (as we're actually not capturing any screenshots, just merely running some tests to ensure they can run if ever needed). Perhaps recently they don't actually execute the Task? I'm not sure. Is it being optimized out as you mentioned?

@Isabel, I don't mind if we also just unschedule these. If we ever revisit screenshots, we can always fix them at the time.

Comment 4

[Aaron Train [:aaronmt]]

We're not sure why these aren't actually running, but since we don't actually have a current need for this Task we'll unschedule it for now. https://bugzilla.mozilla.org/show_bug.cgi?id=1881066

Comment 5

[Geoff Brown [:gbrown]]

Thanks - that works for me.