1085759 - Update base image for flame-kk builds to v188-1

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Nick Thomas [:nthomas] (UTC+12)]

tchung, flame-kk builds are currently on v180. Is v188 ready/good enough to replace it ?

Comment 1

[Tony Chung [:tchung]]

(In reply to Nick Thomas [:nthomas] from comment #0)
> tchung, flame-kk builds are currently on v180. Is v188 ready/good enough to
> replace it ?

not yet.  lets hold off a little longer, because rumor is the GA version is coming soon for the flame.  I give it a week.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Ok, waiting for more info.

Comment 3

[Wesly Huang (EPM)]

Hi Tony and Nick:

Just would like to hear your view, do you think it might be a good idea to update pvt once there is a new SW base released from vendor? Meaning we update to v184/v188 once get it, despite the quality (of course we still expect certain level quality at this stage) and QA test progress?

The pro I see is, we can keep status of pvt and base image aligned, so avoid unnecessary confusion when people see different behavior between based image and pvt full image.

Of course the con/risk is we might downgrade pvt SW quality if vendor does it in base image.

So to me it's a selection that which one we would like to prioritize, pvt SW quality/stability, or the consistency of pvt and base image. How do you think?

btw what's the "GA version" you mentioned, Tony?

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

I'd defer to Tony/QA for most of this. 

For us it's not too much work to update the base image, but we can't promise same day as the request. We can definitely support updating more rapidly than we are now, but hopefully not more than one/twice a week. It would also be possible to make this self-serve for a developer or someone else in the B2G org to do.

Comment 5

[Wesly Huang (EPM)]

Thanks Thomas, in this stage I would say "at most" we might need to update it once a week, and in fact I expect much lower frequency. (maybe twice 2 month or even fewer)

@Tony: would you share your view? I discussed this with some people in TPE and we tend to prioritize the consistency of vendor's base image and pvt SW (after all Flame is more like a development device, than a consumer product), but also would like to hear the opinion from quality before any conclusion.

Comment 7

[Tony Chung [:tchung]]

hi guys, sorry for the late ping.  since my initial "hold off" note till now, a lot of offline discussion happened.  As of this moment, i think we should go ahead and put in v188-1 (UserDebug image) as part of the builds.

There are some specifics that Shawn has brought up (see bug 1088814 to start), that would require the backup files to be updated as part of the build.   ni? on shawn to make sure those files are clear what is needed if we are to repackage v188 and full flashing would cover.  

Once we get that list from Shawn, i am giving the green light for releng to continue this work.

Comment 8

[Shawn Huang [:shawnjohnjr][:shuang] (as a happy gecko contributor)]

Nick knows how to update backup-flame folder for auto build system.
https://bugzilla.mozilla.org/show_bug.cgi?id=1088814#c1
The file lists depend on the script, "device/flame/extract-files.sh".

My intention is to:
1. Update v188 files into backup-flame folder for PVT build
2. Have a better way to indicate current PVT backup-flame folder is using which base image version. Maybe we can have a note into build manifest? (No-Jun mentioned in other mail thread, Bootloader value still said v188, this may confuse people, it doesn't reflect the fact they are using v180 older HAL files).

Comment 9

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

Could we mark the base image library reversion somewhere?
Otherwise I need to compare the library binary size between pvt image and base image.

Comment 10

[Wesly Huang (EPM)]

Hi Nick:

Per comment#7 and comment#8 I guess we are ready to update PVT w/ v188 SW, right? Would you help on that? 

Thank you.

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

Starting on this using UserDebug Image: v188-1.

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [b2g-inbound] Update to v188-1

This adds a 'comment' to the manifest, it's not on device but hopefully it helps people inspect quickly what is being used.

Doc:
 mkdir -p device/tcm
 cd device/tcm
 git clone https://github.com/mozilla-b2g/device-flame flame --branch kitkat
 cd flame
 ./extract-files.sh

 cd ../../..; tar -vcJf backup-flame.tar.xz backup-flame/
 mkdir upload
 mv backup-flame.tar.xz upload
 wget https://github.com/mozilla/build-tooltool/raw/master/tooltool.py
 chmod +x tooltool.py 
 ./tooltool.py distribute --folder upload \
   --message "Bug 1085759, update flame-kk to base v188-1" \
   --user nthomas --host tooltool-uploads.pub.build.mozilla.org \
   --path "/tooltool/uploads/nthomas/pvt"

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

Wesly, tchung - how far should this be uplifted ? All the way to 2.0, or stop at 2.1 ?

Comment 14

[Tony Chung [:tchung]]

(In reply to Nick Thomas [:nthomas] from comment #13)
> Wesly, tchung - how far should this be uplifted ? All the way to 2.0, or
> stop at 2.1 ?

2.0, 2.1, and master is my opinion.   we will still have users on 2.0 that use the flame.

Comment 15

[Tony Chung [:tchung]]

[Blocking Requested - why for this release]:

this is external tracking but needs to be blocking 2.1 because it is necessary for the correct testing enviroment on flame.

Comment 16

[bhavana bajaj [:bajaj]]

Blocking as QA needs this to test correct 2.1 builds.

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8512413 [details] [diff] [review]
[b2g-inbound]  Update to v188-1

https://hg.mozilla.org/integration/b2g-inbound/rev/46d81f906852

Comment 18

[Tony Chung [:tchung]]

Thanks Nick!   by the way, in comment 2, you alluded to showing others how to create self-serve builds like this.   Can we plan something like an education and releng handoff to b2g teams in the future?   I know james lal would be also interested in this discussion.

Comment 19

[Nick Thomas [:nthomas] (UTC+12)]

https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/8f6a875f4987
https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/acddd953156b

Still waiting on a merge to m-c.

Comment 20

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Tony Chung [:tchung] from comment #18)
> Thanks Nick!   by the way, in comment 2, you alluded to showing others how
> to create self-serve builds like this.   Can we plan something like an
> education and releng handoff to b2g teams in the future?   I know james lal
> would be also interested in this discussion.

Anyone with a flame device and access to tooltool could follow the steps in comment #12. For access see
 https://wiki.mozilla.org/ReleaseEngineering/Applications/Tooltool#How_to_enable_a_user_to_run_tooltool_uploads_by_adding_him.2Fher_to_the_vpn_tooltooleditors_LDAP_group

Then you use the tooltool output to create the patch to b2g/config/flame-kk/releng-flame-kk.tt in the gecko repo.

Comment 21

[Florin Strugariu [:Bebe]]

This bug is causing multiple fails on Firefox OS and also breaking automation can we get this patch backed out.

See Bug 1090850 and Bug 1090813 for more info.

Comment 22

[Zac C (:zac)]

We got the error in #21 by flashing our Flames using 'Images' type and using the B2G-flash-tool scripts.

It was replicated both manually and picked up by the device automation that test each build on b2g-i.

Comment 23

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/46d81f906852

Comment 24

[Carsten Book [:Tomcat]]

and backed out per request in #21 and #22 in remote:   https://hg.mozilla.org/mozilla-central/rev/c6989e473f97

Comment 25

[Shawn Huang [:shawnjohnjr][:shuang] (as a happy gecko contributor)]

(In reply to Florin Strugariu [:Bebe] from comment #21)
> This bug is causing multiple fails on Firefox OS and also breaking
> automation can we get this patch backed out.
> 
> See Bug 1090850 and Bug 1090813 for more info.

Viral, are you aware of these problem? With pure v188 image and shallow flash gecko/gaia.

Comment 26

[Nick Thomas [:nthomas] (UTC+12)]

I asked in the dep bugs, but should we backing out on 2.0 and 2.1 too ? Also need advice on how to fix this, outside my area of expertise.

Comment 27

[Robert Chira [:RobertC]]

This issue is also affecting v2.0 and v2.1, can we get a backout on those branches please?

Comment 28

[viral [:viralwang]]

(In reply to Shawn Huang [:shawnjohnjr] from comment #25)
> (In reply to Florin Strugariu [:Bebe] from comment #21)
> > This bug is causing multiple fails on Firefox OS and also breaking
> > automation can we get this patch backed out.
> > 
> > See Bug 1090850 and Bug 1090813 for more info.
> 
> Viral, are you aware of these problem? With pure v188 image and shallow
> flash gecko/gaia.
looks like it relative to full flash, not only gecko/gaia flashing.
maybe we need to check the difference part of v188

Comment 29

[Carsten Book [:Tomcat]]

also backed out in remote:   https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/1b13db1ae45a and
remote:   https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/4c06748190b7

Comment 30

[Tony Chung [:tchung]]

(In reply to Nick Thomas [:nthomas] from comment #26)
> I asked in the dep bugs, but should we backing out on 2.0 and 2.1 too ? Also
> need advice on how to fix this, outside my area of expertise.

viral, what can we do here?  this is a problem, and we need to resolve getting our mozilla images running v188 binaries working over a flame v188 base.

In the meantime, i'm going to advise QA to fall back to testing the current pvtbuilds (that have v180 binaries in them), and test them over a v180 flame base image.   Full flashing current pvtbuilds over v188 is giving us frankenbuilds that we need to stop doing.

or the other option is to shallow flash current pvtbuilds over V188.   which do you suggest is a more realistic solution for testing?

Comment 31

[Nick Thomas [:nthomas] (UTC+12)]

We're going to do a new m-c build, which will contain bug 1034147 and the v188 image, then tchung will run it through jenkins. If all is well we can land in b2g-i after that.

Comment 32

[Tony Chung [:tchung]]

(In reply to Nick Thomas [:nthomas] from comment #31)
> We're going to do a new m-c build, which will contain bug 1034147 and the
> v188 image, then tchung will run it through jenkins. If all is well we can
> land in b2g-i after that.

I was able to test a m-c build from jenkins, which picked up the fix for bug 1034147.  the audio issues reported in bug 1088339, bug 1090813, and bug 1090850, seem to all be fixed now.  however, i can't get dialer to work, and it crashes when trying to dial a phone number.

I'm going to file another bug against the Dialer crash now, and mark it a Qablocker.  We'll need some investigation on why that's happening.   However, i did try full flashing yesterday's pvtbuild (which is agianst v180 bits, and the dialer works).   

I suggest we hold off on merging to b2g-i, until the dialer bug is fixed.  Will file now and add to this bug.

Comment 33

[Tony Chung [:tchung]]

Filed bug 1092002 to track the dialer crash issue.

Comment 34

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8512413 [details] [diff] [review]
[b2g-inbound]  Update to v188-1

Updating checked-in flag for backouts in comment #24 and comment #29.

Comment 35

[Hema Koka [:hema]]

Tony/Nick,

When do we expect this bug to be fixed. There is a 2.1 camera blocker (https://bugzilla.mozilla.org/show_bug.cgi?id=1080912) that has been uplifted/verified yet, because of this bug.

Thanks
Hema

Comment 36

[Tony Chung [:tchung]]

(In reply to Hema Koka [:hema] from comment #35)
> Tony/Nick,
> 
> When do we expect this bug to be fixed. There is a 2.1 camera blocker
> (https://bugzilla.mozilla.org/show_bug.cgi?id=1080912) that has been
> uplifted/verified yet, because of this bug.
> 
> Thanks
> Hema

See https://bugzilla.mozilla.org/show_bug.cgi?id=1092002#c29 for the latest findings.   no ETA yet until we hear back from T2M about the kernel changes.

Comment 37

[Julien Wajsberg [:julienw]]

Should bug 1102972 block this bug?

Comment 38

[Tony Chung [:tchung]]

It sounds like bug 1085759 has resolved this problem.  QA has done some thorough radio/telephony/other tests to validate a try build with these changes are working correctly.   Also, bug 1102682 is fixed and verified against the device.    With that, i suggest we reopen this bug and hopefully get pvtbuilds to work again with this fix.  

Thanks!

Comment 39

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

slight correction :  bug 1092002 was fixed via kernel repo fix bug 1101417

Comment 40

[Tony Chung [:tchung]]

Just a point of confirmation, can we make sure the builds for 2.0, 2.1, and m-c are all affected by this change?

Comment 41

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8512413 [details] [diff] [review]
[b2g-inbound]  Update to v188-1

https://hg.mozilla.org/integration/b2g-inbound/rev/be4ba3d5ca9a
https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/3d2d47bfe3ff
https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/fc90ac008cc1

Comment 42

[:gerard-majax]

Can we make sure we are in sync for our source code on this ?

Comment 43

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/be4ba3d5ca9a

Comment 44

[viral [:viralwang]]

(In reply to Alexandre LISSY :gerard-majax from comment #42)
> Can we make sure we are in sync for our source code on this ?
Partner claimed they have updated for v188 with following list.
device/qcom/common
kernel/lk
kernel/msm
platform/hardware/qcom/camera
platform/hardware/qcom/display
platform/vendor/qcom/msm8610

I'm thinking there are still some missing like recovery still not update-to-date.
My device shows blue on screen if I flash recovery image built in local.
I'm still need to find out the difference to their formal release.

Since we only use kernel and libnfc-nci from partner now, we have no plan to verify the status of all repo :(

Comment 45

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Automated device tests are now full flashing with the pvtbuilds again.