No results in Datazilla for moz-* events due to unable to flash Gonk

RESOLVED FIXED

Status

--
critical
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: Eli, Assigned: davehunt)

Tracking

({perf})

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

4 years ago
The B2G view in Datazilla which by default should display results for moz-app-visually-complete is currently showing no results [1]. This is because the tests are erroring from bug 1048024. This bug has been resolved, but we need to be able to flash Gonk in order to resolve the bug on the test devices. Until the fix can be applied, Datazilla will see no results for these tests.

[1] https://datazilla.mozilla.org/b2g
(Assignee)

Comment 1

4 years ago
As I understand it a single full flash should rectify this. I'm currently working on doing that and will report back.
Assignee: nobody → dave.hunt
Status: NEW → ASSIGNED
Flags: needinfo?(dave.hunt)
(Assignee)

Comment 2

4 years ago
This looks like it's fixed now, results have started to appear.
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Flags: needinfo?(dave.hunt)
Resolution: --- → FIXED
(Assignee)

Comment 3

4 years ago
This has returned due to a recent request to reflash the automation devices with v188-1 and to subsequently only shallow flash them. I want all devices in automation to be consistent, and therefore want to avoid a one-off full flash on only the perf devices. If we know what partition(s) need to be flashed then perhaps we can find out if these would affect the functional tests, and if not we can only flash those.

Tony: Please advise how we can keep all automation devices consistent but restore performance testing coverage.
Status: RESOLVED → REOPENED
Flags: needinfo?(tchung)
Resolution: FIXED → ---
(Assignee)

Comment 4

4 years ago
Fabrice: Do you know which partitions would need to be flashed for the fix from bug 1048024? If it turns out that these will not introduce the issues that Tony is trying to avoid then it would be safe for us to flash them across all devices.
Flags: needinfo?(fabrice)
Dave, did you get what you need from dhylands?
Flags: needinfo?(fabrice)
(Assignee)

Comment 6

4 years ago
Yes, I think so. I've commented on bug 1092002 and set it as a blocker.
Depends on: 1092002

Comment 7

4 years ago
(In reply to Dave Hunt (:davehunt) from comment #3)
> This has returned due to a recent request to reflash the automation devices
> with v188-1 and to subsequently only shallow flash them. I want all devices
> in automation to be consistent, and therefore want to avoid a one-off full
> flash on only the perf devices. If we know what partition(s) need to be
> flashed then perhaps we can find out if these would affect the functional
> tests, and if not we can only flash those.
> 
> Tony: Please advise how we can keep all automation devices consistent but
> restore performance testing coverage.

sorry for the delayed response.  i agree we need to sync everything together, and hopefully you can found the partitions you need removed.  v188-1 is still going to be the best image (and final image i'm told from TAMs), that we'll need to use for all automation.   until we can get the pvtbuilds to carry the working v188 binaries, i'm concerned we really shouldnt be full flashing.
Flags: needinfo?(tchung)
(Assignee)

Comment 8

4 years ago
We haven't had any automated performance results for over 10 days now, and it doesn't look like bug 1092002 has a clear indication of what's needed or when it will be resolved. In order to restore performance test coverage can we either revert to using v180 (or v184) and full flashes, or to use v188-1 and full flashes (excluding the releng boot.img).

I'm planning to change the way we flash devices in our CI to a single combined flash of the base images, substituting any images produced by our own builds. The latter is usually boot.img, system.img, and userdata.img, however considering our own boot.img appears to be the cause of these issues when combined with v188 base builds, I am suggesting we just use the version in the base build.
Flags: needinfo?(tchung)

Comment 9

4 years ago
(In reply to Dave Hunt (:davehunt) from comment #8)
> We haven't had any automated performance results for over 10 days now, and
> it doesn't look like bug 1092002 has a clear indication of what's needed or
> when it will be resolved. In order to restore performance test coverage can
> we either revert to using v180 (or v184) and full flashes, or to use v188-1
> and full flashes (excluding the releng boot.img).
> 
> I'm planning to change the way we flash devices in our CI to a single
> combined flash of the base images, substituting any images produced by our
> own builds. The latter is usually boot.img, system.img, and userdata.img,
> however considering our own boot.img appears to be the cause of these issues
> when combined with v188 base builds, I am suggesting we just use the version
> in the base build.

can you elaborate on "own" builds?   these would not be pvtbuilds, would they?   if the scenario you mentioned gets you full flashing builds on v188-1, then i would suggest going with that route until we resolve 1092002.  I agree, lets not wait much longer and get perf data running again since it takes time to build up the queue.
Flags: needinfo?(tchung) → needinfo?(dave.hunt)
(Assignee)

Comment 10

4 years ago
(In reply to Tony Chung [:tchung] from comment #9)
> can you elaborate on "own" builds?   these would not be pvtbuilds, would
> they?   if the scenario you mentioned gets you full flashing builds on
> v188-1, then i would suggest going with that route until we resolve 1092002.
> I agree, lets not wait much longer and get perf data running again since it
> takes time to build up the queue.

They would be pvtbuilds, as this is what we're currently using for our on-device test automation. I will see what I can do today to unblock our performance coverage.
Flags: needinfo?(dave.hunt)
(Assignee)

Comment 11

4 years ago
Tony: With bug 1092002 now resolved, can you confirm it's safe to go back to full flashing in automation?
Flags: needinfo?(tchung)
(In reply to Dave Hunt (:davehunt) from comment #11)
> Tony: With bug 1092002 now resolved, can you confirm it's safe to go back to
> full flashing in automation?

yep, thats the plan.  I've requested this work resume again in https://bugzilla.mozilla.org/show_bug.cgi?id=1085759#c38.
Flags: needinfo?(tchung)
(Assignee)

Comment 13

4 years ago
I've switched the performance tests back to performing a full flash with the pvtbuilds. If this is successful I'll do the same for the UI tests.
Flags: needinfo?(dave.hunt)
(Assignee)

Comment 14

4 years ago
We have test results again!
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Flags: needinfo?(dave.hunt)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.