Closed Bug 885114 Opened 11 years ago Closed 7 years ago

[tracking] 1.0->1.1 OTA data migration testing issues

Categories

(Firefox OS Graveyard :: General, defect)

Product:
Firefox OS Graveyard
Component:
General
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: akeybl, Assigned: askeing)

References

Details

(Keywords: meta) 

Tony mentioned Taipei was going to test this - we need to make sure profile migration issues are not latecomers.
bweng - can you help with this? Also see the "Copying v1.0.1 profile to v1.1 supported?" thread on dev.b2g for possible pitfalls.
Flags: needinfo?(bweng)
i've talked to bruce yesterday, and he agreed to the twqa team to investigate this.
I have talked to Al and Askeing about this Gecko/Gaia OTA test. There is what we will do. 

1.	Searching for parameter and build a full package – Done, but Askeing is checking if he does this correctly about the mar wrapper. 
2.	Run the diff and determine what are the new files are required to update 
3.	Modify the update link.
4.	Testing.

However, for a complete FOTA, we need system engineer to confirm if that is doable or not(This needs system engineer schien to update)
Flags: needinfo?(bweng) → needinfo?(schien)
We cannot generate FOTA package because we cannot build boot image from our B2G repository.
Flags: needinfo?(schien)
i think as long as you don't flash the userdata, you can test data migration from v1.0.1 to v1.1.0, which is a big part of a major upgrade from one version to the next
(In reply to Joe Cheng [:jcheng] from comment #5)
> i think as long as you don't flash the userdata, you can test data migration
> from v1.0.1 to v1.1.0, which is a big part of a major upgrade from one
> version to the next

As long as OTA/flash update testing is applicable to FOTA, I'm OK with that.
I thought the ril wouldn't be the same because MMS was introduced in SMS in 1.1.  You would have to OTA the ril as well, I think else you may crash.
quick test result here:

Inari
from v1.0.1 (pvt)
    Gaia:     93241eb6c5d6c110710fad8da3ccd4423312b0c9
    Gecko:    http://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/rev/9c62297d11b0
    BuildID   20130624070215
    Version   18.0
to v1-train (complete gaia/gecko OTA update)
    Gaia mozillaorg/v1-train - 16b872bd7d3aa4f1a7ae73bb19806af8b5d23dc0
    Gecko mozillaorg/v1.1.0hd - 914243b75089d42b0672c8b01b7adf33568309f5

Calendar: Blank screen when launch App after OTA.
Message: No original messages after OTA.
Email: No email. But still have original account settings.

Settings: ok
    Change "Display" to no auto adjust, and screen never timeout.
    Still have original settings.
Contacts: ok
    Add contacts.
    Still have original contacts.
Clock: ok
    Add alerts.
    Still have original alerts.
Browser: ok
    Connect to some websites, bookmark some website.
    Still have original history and bookmark list.
Bookmark on Homescreen: ok
    Bookmark some websites on Homescreen.
    Still have original bookmarks on Homescreen.
FMRadio: ok
    Add favorite sites.
    Still have original favorite sites.
Gallery: ok
    Take some photos/videos by camera.
    Still have original photos/videos.
Video: ok
    Take some videos by camera.
    Still have original videos.
Music: ok
    Add musics.
    Still have original musics.
Call log: ok
    Call other phone, check the call log exist.
    Still have original call log.
Usages: ok
    Access network by 3G or Wifi, check the size of usages.
    The size of usages increase.
Downloaded Apps: ok
    Download Apps from Marketplace.
    Still show on Homescreen.
(In reply to Askeing Yen[:askeing] from comment #8)
> Inari
> to v1-train (complete gaia/gecko OTA update)
>     Gaia mozillaorg/v1-train - 16b872bd7d3aa4f1a7ae73bb19806af8b5d23dc0
>     Gecko mozillaorg/v1.1.0hd - 914243b75089d42b0672c8b01b7adf33568309f5

The command of building complete gaia/gecko OTA update package:
B2G_UPDATER=1 B2G_SYSTEM_APPS=1 MOZ_BUILD_DATE=XXX ANDROIDFS_DIR=XXX ./build.sh gecko-update-full
(In reply to Askeing Yen[:askeing] from comment #8)
> quick test result here:
> 
> Inari
> from v1.0.1 (pvt)
>     Gaia:     93241eb6c5d6c110710fad8da3ccd4423312b0c9
>     Gecko:   
> http://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/rev/9c62297d11b0
>     BuildID   20130624070215
>     Version   18.0
> to v1-train (complete gaia/gecko OTA update)
>     Gaia mozillaorg/v1-train - 16b872bd7d3aa4f1a7ae73bb19806af8b5d23dc0
>     Gecko mozillaorg/v1.1.0hd - 914243b75089d42b0672c8b01b7adf33568309f5
> 
> Calendar: Blank screen when launch App after OTA.
> Message: No original messages after OTA.
> Email: No email. But still have original account settings.
Update:
ActiveSync, m.google.com
Inbox cannot see the emails, but All Maill can see the emails correctly.
Unagi
from v1.0.1 pvt
    Gaia:     93241eb6c5d6c110710fad8da3ccd4423312b0c9
    Gecko:    http://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/rev/9c62297d11b0
    BuildID   20130626070217
    Version   18.0
to v1-train
    Gaia mozillaorg/v1-train - 4d6cbe49fdba32f72ec8bc6280e4782c250ab41e
    Gecko mozillaorg/gecko-18 - 914243b75089d42b0672c8b01b7adf33568309f5

Same with comment 8.
Inari
from v1-train pvt 
    Gaia:     477e57212d4338e87182d9a8be96bb5c59c0a792
    Gecko:    http://hg.mozilla.org/releases/mozilla-b2g18/rev/6a9ca50a313b
    BuildID   20130626230209
    Version   18.0
to v1-train
    Gaia mozillaorg/v1-train - 16b872bd7d3aa4f1a7ae73bb19806af8b5d23dc0
    Gecko mozillaorg/gecko-18 - 914243b75089d42b0672c8b01b7adf33568309f5


Calendar: ok
    Create events.
    Still have original events.
Message: ok
    Send/receive messages.
    Still have original messages.
Email: ok
    Set up the email account.
    Still have original email settings and email.

Settings: ok
    Change "Display" to no auto adjust, and screen never timeout.
    Still have original settings.
Contacts: ok
    Add contacts.
    Still have original contacts.
Clock: ok
    Add alerts.
    Still have original alerts.
Browser: ok
    Connect to some websites(history), bookmark some website.
    Still have original history and bookmark list.
Bookmark on Homescreen: ok
    Bookmark some websites on Homescreen.
    Still have original bookmarks on Homescreen.
FMRadio: ok
    Add favorite sites.
    Still have original favorite sites.
Gallery: ok
    Take some photos/videos by camera.
    Still have original photos/videos.
Video: ok
    Take some videos by camera.
    Still have original videos.
Music: ok
    Add musics.
    Still have original musics.
Call log: ok
    Call other phone, check the call log exist.
    Still have original call log.
Usages: ok
    Access network by 3G or Wifi, check the size of usages.
    The size of usages increase.
Downloaded Apps: ok
    Download Apps from Marketplace.
    Still show on Homescreen.
Depends on: 887698
Depends on: 887700
Depends on: 887701
Askeing, thanks for the testcases.  but i'm going to assume this is all OTA testing.  Were we ever able to set up a FOTA server in house?   comment 4 says otherwise.

ultimately, the FOTA updating will also tell us if RIL and Firmware changes would break the experience.   And I dont have the answer to how to set this up.
(In reply to Tony Chung [:tchung] from comment #13)
> Askeing, thanks for the testcases.  but i'm going to assume this is all OTA
> testing.  Were we ever able to set up a FOTA server in house?   comment 4
> says otherwise.
> 
> ultimately, the FOTA updating will also tell us if RIL and Firmware changes
> would break the experience.   And I dont have the answer to how to set this
> up.

I believe the FOTA tests here cover gaia and gecko update only.
We may need to know how release engineering team generates full patch and we can generate the diff files for FOTA testing.
FYI in relation to the testing going on here - make sure you are including logcat during testing. Most of the dependencies here are going to tough for developers to action without the right logging information.
(In reply to Al Tsai [:atsai] from comment #14)
> (In reply to Tony Chung [:tchung] from comment #13)
> > Askeing, thanks for the testcases.  but i'm going to assume this is all OTA
> > testing.  Were we ever able to set up a FOTA server in house?   comment 4
> > says otherwise.
> > 
> > ultimately, the FOTA updating will also tell us if RIL and Firmware changes
> > would break the experience.   And I dont have the answer to how to set this
> > up.
> 
> I believe the FOTA tests here cover gaia and gecko update only.
> We may need to know how release engineering team generates full patch and we
> can generate the diff files for FOTA testing.

thats what i presume, you tested OTA, not FOTA.   the testing you guys did is legitimate, but we really need the full stack in order to mimic what OEMs would do.  can you work with dhylands to see what's needed to set that up?   (also Com RIL would be highly affected for FOTA, which i dont know how to set that up -- may need help from mvines team)
Building FOTA package need obtain boot.img and recovery.fstab from our ODM/OEM partner. This implies we'll get some false alarm on FOTA update due to not using latest version of these two proprietary files. Creating a auto-sync/version system for these proprietary files will mitigate the problem.
(In reply to Tony Chung [:tchung] from comment #16)
> (In reply to Al Tsai [:atsai] from comment #14)
> > I believe the FOTA tests here cover gaia and gecko update only.
> > We may need to know how release engineering team generates full patch and we
> > can generate the diff files for FOTA testing.
> 
> thats what i presume, you tested OTA, not FOTA.   the testing you guys did
> is legitimate, but we really need the full stack in order to mimic what OEMs
> would do.  can you work with dhylands to see what's needed to set that up?  
> (also Com RIL would be highly affected for FOTA, which i dont know how to
> set that up -- may need help from mvines team)

Base on comment 17 and have the discussion with teammates, there are something needs partner and Release Engineer's help.

### gaia/gecko OTA update ###
After trace the pvtbuilds' log, I found the pvtbuilds server will build gaia/gecko update file (objdir-gecko/dist/b2g-update/b2g-gecko-update.mar). But the update file do not be published.
So we need Release Engineer's help to publish the update file (b2g-gecko-update.mar) and update.xml file.
The OTA update.xml can be create by following command:
<B2G_DIR>$ ./tools/update-tools/build-update-xml.py -o update.xml -i <BUILD_ID> -u http://<URL_TO_SERVER>/b2g-gecko-update.mar -v <APP_VER> -V <PLATFORM_VER> -c objdir-gecko/dist/b2g-update/b2g-gecko-update.mar

### FOTA update ###
We need partner's help to provide the boot.img and recovery.fstab files to generate the FOTA package and update.xml file. Then publishing them to pvtbuilds server.
The FOTA package and update.xml can be created by following command:
<B2G_DIR>$ ./tools/update-tools/build-fota-mar.py -o update.mar out/target/product/$DEVICE/$DEVICE-ota-$VARIANT.$USER.zip
<B2G_DIR>$ ./tools/update-tools/build-update-xml.py -O -o update.xml -i <BUILD_ID> -u http://<URL_TO_SERVER>/update.mar -v <APP_VER> -V <PLATFORM_VER> -c update.mar
Assignee: nobody → fyen
Adding Dave Hylands and partner email to try and pull more people into this discussion.
Flags: needinfo?(jaeohkim83)
Flags: needinfo?(dhylands)
Depends on: 840849
So after reading this, it isn't clear to me what the question is?

As to differences between FOTA/OTA, there are different updaters which perform the work, and FOTA is capabale of updating more stuff.

For example, it's possible for FOTA to repartition, format partitions, and other stuff along those lone. If the FOTA update wipes the data partition, then of course, all user data will disappear.

If you assume that the FOTA doesn't touch /data partition, then from the data migration perspective, there really isn't much difference between FOTA/OTA.
Flags: needinfo?(dhylands)
Due to the Ikura is one of target devices, we should generate Ikura builds if possible. (Bug 840849)
(In reply to Askeing Yen[:askeing] from comment #21)
> Due to the Ikura is one of target devices, we should generate Ikura builds
> if possible. (Bug 840849)

Hi Askeing, can you not use the current inari builds for this right now?   The biggest concerns i have is whether or not we can get a FOTA update (inari with commercial RIL), and verify if data migration is working or busted. 

Re:  boot.img and recovery.fstab , whats the latest status on this?
(In reply to Dave Hylands [:dhylands] from comment #20)
> If you assume that the FOTA doesn't touch /data partition, then from the
> data migration perspective, there really isn't much difference between
> FOTA/OTA.

Let's make this assumption, and verify with partners. Bhavana, can you add FOTA data migration to the partner checklist?

I agree with tchung on using whatever device suits us best currently. No need to block on bug 840849.
Flags: needinfo?(jaeohkim83) → needinfo?(bbajaj)
Summary: 1.0->1.1 OTA/FOTA data migration testing → 1.0->1.1 OTA data migration testing
(In reply to Alex Keybl [:akeybl] from comment #23)
> (In reply to Dave Hylands [:dhylands] from comment #20)
> > If you assume that the FOTA doesn't touch /data partition, then from the
> > data migration perspective, there really isn't much difference between
> > FOTA/OTA.
> 
> Let's make this assumption, and verify with partners. Bhavana, can you add
> FOTA data migration to the partner checklist?
Done, Its added to the wiki : https://wiki.mozilla.org/Firefox_OS_Guidelines_For_Partners#Certification_Process under the partner certification checklist.
> 
> I agree with tchung on using whatever device suits us best currently. No
> need to block on bug 840849.
Flags: needinfo?(bbajaj)
(In reply to Tony Chung [:tchung] from comment #22)
> (In reply to Askeing Yen[:askeing] from comment #21)
> > Due to the Ikura is one of target devices, we should generate Ikura builds
> > if possible. (Bug 840849)
> 
> Hi Askeing, can you not use the current inari builds for this right now?  
> The biggest concerns i have is whether or not we can get a FOTA update
> (inari with commercial RIL), and verify if data migration is working or
> busted. 
There are two type testing:
 a)Data Migration.
   Test the device should keep the user data and the app can load the data correctly after update from v1.0.1 to 1.1.0.
 b)Functionality.
   Test the functionality of device works fine after update from 1.0.1 to 1.1.0.
If we focus on a), just like dhylands said at comment 20, there isn't much diff between FOTA/OTA.
And if we want to do b), we have to test updating ril by FOTA. 

Base on your comment, I think we want to do both a) and b), right?

> Re:  boot.img and recovery.fstab , whats the latest status on this?
Kevin,
Could you help to get boot.img and recovery.fstab of Inari from partnere?
Thank you.
Flags: needinfo?(khu)
TLDR. Just want to double confirm. Are we asking for Inari or Ikura? Ikura is the final shipping device. Thanks.
Flags: needinfo?(khu) → needinfo?(fyen)
Base on comment 22 and comment 23, we have to select one device which suits us best currently. So Inari should be the better choice now.

If possible, getting files for both Inari/Ikura is the best.

Thank you so much.
Flags: needinfo?(fyen) → needinfo?(khu)
Okay. Ivan, can you help to talk with the partner and ask for it? Thanks.
Flags: needinfo?(khu) → needinfo?(itsay)
The files of boot.img and recovery.fstab for Ikura can be obtained from partner's FTP server. I've sent the FTP server info through another email thread. Please check it. For the Inari ones, I am still waiting for the response from our partner.
Flags: needinfo?(itsay)
Update:
Partner will take care the testing of upgrading from one version to another.
So we can just focus on testing update of Mozilla part (Gaia/Gecko OTA).
There is the way to test OTA update from pvtbuilds v1.0.1 to v1.1.0.
  1. Flash v1.0.1 build from pvtbuilds server.
  2. Change the update url from v1.0.1 to v1.1.0 nightly. Then reboot the device.
  3. Check the version of B2G is v1.0.1. (Settings > Device Info > More Info)
  4. Do something on v1.0.1 build.
  5. Install OTA update to v1.1.0.
  6. Check the version of B2G is v1.1.0. (Settings > Device Info > More Info)
  7. Do something on v1.1.0 build.
Note that you should open "Settings > Device Info > More Info > Developer > Remote Debugging" when you need to see logcat from the device.
Depends on: 891797, 892392
No longer depends on: 840849
No longer depends on: 893709
Depends on: 892399, 893709
Whiteboard: [NPOTB]
Actually - I'm going to switch this to a tracker bug - we're using this right now to track all bugs we find from OTA 1.01 --> 1.1 testing.
blocking-b2g: leo+ → ---
Keywords: meta
Summary: 1.0->1.1 OTA data migration testing → [tracking] 1.0->1.1 OTA data migration testing issues
Whiteboard: [NPOTB]
Depends on: 897813
Depends on: 907160
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.