1037592 - [OTA] Need a way to test other builds than latest

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Peter Bylenga [:PBylenga]]

Can we have a way to update a build to a specific build rather than OTA to latest?  This is needed to prove regressions and do regression window hunting for issues found for OTA flow.

Our current STR to do 1.4 to 2.0:

1. setup flame with 1.4 gaia + gecko + v122 base
2. change channel to aurora with b2g-flash-tool script
3. change URL to https://aus4.mozilla.org/update/3/B2G/28.0/20140709000201/flame/en-US/aurora/Boot2Gecko%202.0.0.0-prerelease/default/default/update.xml?force=1

We also test 1.3 to 2.0 and 2.0 to 2.1 currently.

Comment 1

[Peter Bylenga [:PBylenga]]

Please needinfo me if you need any other information.

Comment 2

[[e-mail] (:bhearsum)]

What's wrong with flashing? The updater isn't meant to be able to give you arbitrary updates from X to point Y - it's meant to keep you up to date. I don't know of a way to implement this with it, either.

For similar regrission hunting on desktop, people have written bisection stricts to help them out. Maybe something similar would help you here?

Comment 3

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Hi Ben,

The updater will keep the data, whereas our flashing mechanism will not keep the data.  We end up having to backup the profile and then push it back or or otherwise and it might not lead to finding the same bug.

If there's an easy way to similuate update regression hunting, it would be appreciated;
Is there a way to copy the mar file onto sdcard and then have the updater run off of the mar file?

Maybe this might be a question for dhylands...?

Comment 4

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Oh.  Duh.  I found that we can just pull the mar file down based on url.

Comment 5

[Dave Hylands [:dhylands]]

(In reply to Naoki Hirata :nhirata (please use needinfo instead of cc) from comment #3)
> Hi Ben,
> 
> The updater will keep the data, whereas our flashing mechanism will not keep
> the data.  We end up having to backup the profile and then push it back or
> or otherwise and it might not lead to finding the same bug.
> 
> If there's an easy way to similuate update regression hunting, it would be
> appreciated;
> Is there a way to copy the mar file onto sdcard and then have the updater
> run off of the mar file?
> 
> Maybe this might be a question for dhylands...?

When I test updates locally, we fake the update.xml file to say that the buildid of the update is for version 99 (IIRC).

Then it always considers the current version to be out-of-date and will download and apply the update.mar file.

So if you can control what goes on the server then you could play a similar game. It's pretty straight forward to run your own server. python has a webserver builtin, and I've used that for serving up updates:
http://www.linuxjournal.com/content/tech-tip-really-simple-http-server-python

Comment 6

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

Ok, so given that we can pull builds via downloading the mar file and then also hosting the file up with a faked xml to update, it's possible to do all this currently without any changes.

I was hoping for some easier mechanism...  Thanks for the info dhylands!

Comment 7

[[e-mail] (:bhearsum)]

nthomas pointed out to me that we can use special update channels to do this if you only care about specific builds. Ie, if there's a few specific MARs you want to update to, we can set up special channels for that. Each one of these requires someone from RelEng to poke around to make it happen though, so it's probably not something we can do if you need a large quantity of them, or you need quick turnaround at various times.

It sounds like you may have a workaround already, though.