837769 - [B2G][Settings] Gaia commit after OTA and manual flash are mismatching

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Natalya Kot [:nkot]]

Attachment: after OTA

Description:
*Gaia commit after OTA and manual flash are mismatching, Gaia version is not starting with 04e3e in the sources.xml

Repro:
Prerequisites:
*Have a regular (no special ril) previous daily build on Test Unagi (mine was the 31st regular)
*Set WiFi to On
*Set Remote debugging to On: Settings => Device info => More info =>     Developer

1. Check for updates: Settings => Device info => Software updaates => Check now
2. Download the latest update OTA: pull out notification bar to locate the System Update => select to download
3. When installed, make a note or screenshot of the Gaia info:
       Settings => Device info => More info => Git commit info 
4. Flash unagi with the latest nightly/B2G (download from Mozilla)
5. When installed, make a note or screenshot of the Gaia info:
       Settings => Device info => More info => Git commit info 


Expected:
*Git Commit number should be the same after the OTA and manual flash

Actual:
*Git Commit numbers are different: 
OTA:   04e3e923bdae0773b0133ffa4958831...
Flash: b277f372f097c77dd2ec1988a80e458280dda491

Repro frequency:
Occurred on 3 different Unagi devices

Screenshots attached

Notes:
*Git Commit number and the build ID are not in sync
See bug # 837170

Comment 1

[Natalya Kot [:nkot]]

Attachment: flash

Comment 2

[Marcia Knous [:marcia]]

I saw this on one device when testing today as well. I had to perform a second OTA update to get on today's build.

Comment 3

[Dave Hylands [:dhylands]]

It's worth mentioning, that once an OTA starts on a particular version, it will "stick" with that download until the update has been fully downloaded and applied.

So, if your unagi loses its connection with wifi partway through a download, when it eventually gets reconnected, even if a newer OTA version is available, it will continue to download the older version.

This behaviour is intentional. See bug 831701 for details.

So, if you really want to make sure that your unagi is on the latest, then after an update is applied, do a "Check Now" for new updates. If nothing shows up within 10 minutes, then you've got the latest version.

Comment 4

[Tony Chung [:tchung]]

Ive reproduced this today, when doing an OTA update to the latest.   The buildID matches, but the gaia commits are off.

It's showing up ad 2013-01-31.   but today is 02-05.  Are we not syncing 

On device:
Gecko  http://hg.mozilla.org/releases/mozilla-b2g18_v1_0_0/rev/c39d81e7a2ff
Gaia   c4589884c89f724ba0f72107968d4111d4a9f1b2
BuildID 20130205070201
Version 18.0

Nightly update:
<updates><update type="minor" appVersion="18.0" version="18.0" extensionVersion="18.0" buildID="20130205070201" licenseURL="http://www.mozilla.com/test/sample-eula.html" detailsURL="http://www.mozilla.com/test/sample-details.html"><patch type="complete" URL="http://update.boot2gecko.org/nightly/b2g_update_20130205070201.mar?build_id=20130205070201&version=18.0" hashFunction="SHA512" hashValue="d40cfe9a764ac5527601559bdf1412e85f4ff908fbbad9f458975dde9bdf97afeeed995e55e55b285f3d036555666c3f69a35927f58a34ab71980a9df2c6ca58" size="51187790"/></update></updates>

Comment 5

[Tony Chung [:tchung]]

Attachment: screenshot of 1/31 gaia commit date

Comment 6

[Marcia Knous [:marcia]]

I see the same thing as tchung with my update today. 

I will note on the phone that updated, the update did fail the first time with a message that it had trouble downloading the update. I then restarted the update and when the phone had finished I have the same thing showing as Comment 5.

Comment 7

[Tony Chung [:tchung]]

+ more releng folks.

This is a build blocker, in that having gaia/gecko out of sync bits is invalidating our nightly testing.

Comment 8

[Aki Sasaki (not active)]

(In reply to Tony Chung [:tchung] from comment #4)
> Ive reproduced this today, when doing an OTA update to the latest.   The
> buildID matches, but the gaia commits are off.
> 
> It's showing up ad 2013-01-31.   but today is 02-05.  Are we not syncing 
> 
> On device:
> Gecko  http://hg.mozilla.org/releases/mozilla-b2g18_v1_0_0/rev/c39d81e7a2ff
> Gaia   c4589884c89f724ba0f72107968d4111d4a9f1b2
> BuildID 20130205070201
> Version 18.0

Why are you on b2g18_v1_0_0?  Aren't we dogfooding/testing b2g18 aka v1-train aka 1.0.1 now?

Comment 9

[Julien Wajsberg [:julienw]]

Well, which branches the nightlies are generated from ?

Comment 10

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

OTA is pulling the v1.0.0... that's from the build right?

Is the build suppose to be pulling from  v1.0.0 or from v1-train? or from v1.0.1?

Comment 11

[Marcia Knous [:marcia]]

Attachment: Screenshot from freshly flashed phone

Tony asked me to add the info from my freshly flashed device - I ran the check version script and it shows the following:

Gecko  http://hg.mozilla.org/releases/mozilla-b2g18/rev/2ae874433a14
Gaia   5f97bb66a70e0e61b1604f2572dc88cbbc140a10
BuildID 20130205070201
Version 18.0

Comment 12

[Tony Chung [:tchung]]

We think we figured it out.   doing OTA update remains on v1.0.0, while flashing puts you on v1.0.1.   here's alex's email investigation:

-----------------------------------
We took the changeset and found http://git.mozilla.org/?p=releases/gaia.git;a=commit;h=c4589884c89f724ba0f72107968d4111d4a9f1b2, which pointed us at 833213 – [Email] Search breaks on empty subject; visible in "subject" and "all" search filters. It is true that these changes were uplifted today to v1.0.0/v1-train, so the date is associated with the initial commit to master of the tip changeset.

If the updates are being created from v1-train, the changeset should be 5f97bb66a70e0e61b1604f2572dc88cbbc140a10 (see https://bugzilla.mozilla.org/show_bug.cgi?id=833213#c14). Flashing the build at https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-b2g18-unagi/ appears to give Marcia 5f97bb66a70e0e61b1604f2572dc88cbbc140a10. Tony's OTA gets c4589884c89f724ba0f72107968d4111d4a9f1b2. Are we providing updates to v1.0.0 instead of v1.0.1 (v1-train)?
-----------------------------------

So the bug i think is for nightly builds pre 1/25 that has been OTA'ing, we're stuck on v1.0.0 branch; but should be v1.0.1.   Also, the date/time looks old, but it is the correct time of the merge.   I think we should just remove that gaia commit info date, and i'll file a seperate bug for it.

Comment 13

[Aki Sasaki (not active)]

Attachment: stop generating updates for v1_0_{0,1}

Can also remove the lines.

The problem here is we were generating updates for each branch.

Comment 14

[Chris AtLee [:catlee]]

Comment on attachment 710367 [details] [diff] [review]
stop generating updates for v1_0_{0,1}

Review of attachment 710367 [details] [diff] [review]:
-----------------------------------------------------------------

I'd delete the lines rather than comment them out. r+ either way

Comment 15

[Aki Sasaki (not active)]

Comment on attachment 710367 [details] [diff] [review]
stop generating updates for v1_0_{0,1}

deleted: http://hg.mozilla.org/build/buildbot-configs/rev/c7499e91024d

Comment 16

[Rail Aliiev [:rail]]

in production

Comment 17

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(moving to correct component, now that the root cause is known and being worked.)

These changes were rolled out in a reconfig just now.

Comment 18

[Aki Sasaki (not active)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=19473222&tree=Mozilla-B2g18_v1_0_0&full=1 skips the upload-updates step.
We're good here.

Comment 19

[Lukas Blakk [:lsblakk] use ?needinfo]

If you need to uplift this fix to branches please go ahead and nominate, we'll approve them but this bug is not a shipping blocker.

Comment 20

[Natalya Kot [:nkot]]

Fix verification,

Preconditions:
Unagi was manually flashed to build ID:2013-02-06-110509
Gaia commit: 9f5a4a9f6d321dbb4805f1f98623e770b0e89698

*Issue did not repro in Unagi build ID:2013-02-07-070202
both OTA and manually flashed Gaia versions were matching
Gaia commit: 7e54ca673277b20b1d91d18477dc44d6ad226761


*Did not repro in Unagi build ID: 2013-02-08-070201
both OTA and manually flashed Gaia versions were matching 
Gaia commit: cff23a80c41b7de223d27c6a6f1f82f95c9c5f6b