Closed Bug 1014632 Opened 10 years ago Closed 9 years ago

FlatFish: Integrate boot.img and kernel compilation into the build system

Categories

(Release Engineering :: General, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: _AtilA_, Assigned: _AtilA_)

References

Details

(Whiteboard: [flatfish][TCP=feature-p1])

Attachments

(2 files)

Sounds like a good idea to integrate the creation of boot and recovery images into the build system, so, for example, we would be able to build OTA/FOTA packages.
Component: Release Automation → General Automation
QA Contact: bhearsum → catlee
Assignee: nobody → atilag
Just a status update for those Flatfishers who are willing this bug to be closed.
I'm still working on it, I have already implemented kernel build into the build system, so a patch will come very soon.
We have finally discovered a way to recover bricked devices, and bad generated images is one of the best ways so brick tablets, so now I can retake this bug and finalize it.
I'm getting all the images as a result of the build, but something is wrong with the ramdkisk (I think), just keep working on it.
Whiteboard: [Flatfish only] → [flatfish]
Ok, I have all patches ready to get reviewed, but we need to take some steps first so I would like to ask for help on how to proceed.

1 - We need to keep a fork of the kernel with all the patches needed for flatfish applied, and with a new Android makefile that will actually tells the build system how to compile the kernel. I already did that and it's all under my github account: https://github.com/atilag/flatfish-kernel.git. Maybe Mozilla should fork it.

2 - We need a prebuilt toolchain to compile the kernel. I used the linaro's (based on gcc-4.6) and uploaded it to my github account too: https://github.com/atilag/gcc-linaro-arm-linux-gnueabi-2012-04-20120426_linux.git. Google's toolchain didn't work.

3 - Actual device configuration is downloaded from a github account owned by the vendor: https://github.com/flatfish-fox/flatfish-device_allwinner_common.git , I need to update some files there, so we have two options: I can make a pull-request to theirs repository, so they will have to review my changes, or we can fork it and make a pull-request there, so Michael Wu can review them. What's the best option here?

After decide where we put all this stuff, we have to update the manifest file accordingly.

Sorry if I require information from someone that shouldn't, I'm not really sure who are the correct people for this platform.

Thanks!
Flags: needinfo?(mwu)
Flags: needinfo?(asa)
As Michael suggested me, I could modify only two Makefiles in the kernel in order to compile with google toolchain (arm-eabi-). So there's no need to add an extra toolchain.
I updated my kernel fork with these changes.
Points 1 and 3 still need an action to be taken.

I love the idea to keep our own forks of kernel and device-config. It makes thing a lot of easier for future developments. So if Mozilla forks them from my account, I can modify the flatfish manifest file to point to the correct repositories.

Is that ok?
If the vendor isn't doing active development, forking is fine with me. If they are, we should figure out the best way to collaborate with them. I haven't been involved with flatfish much so I have no idea what the best thing is here.
Flags: needinfo?(mwu)
Joe Cheng maintains our partner relationship so he can probably say more.
Flags: needinfo?(asa)
Ok, requesting some advices from Joe :)
Flags: needinfo?(jcheng)
Blocks: flatfish
Whiteboard: [flatfish] → [flatfish][TCP=feature-p1]
redirecting ni request to Jenny
Flags: needinfo?(jcheng) → needinfo?(jeliu)
Our partner is going to be slow on development but they could review and make changes if needed

The purpose of this bug is to make OTA available for Flatfish
I wonder if you are offering to provide the machines to build OTA packages and hosting the servers?

If so, it is also Mozilla’s plan to be doing so, even though it is taking some time
We can include flatfish partner in the bug to consider the changes you are proposing
Nop, I didn't want to provide any hardware/hosting for OTA, the intention of the bug was to make "complete" builds possible, so Mozilla can build and provide OTA/FOTA updates as they do with other platforms like Flame.
Ok, I'm gonna try to apply my patches to their repositories and talk to them so they can provide a copy of the kernel with all the patches in github (and update the manifest to point to this repo).

We DO need to add builds for this platform in the try-server, because they are broken very often :(
Regarding to comment 9 and comment 10, yes, Mozilla's plan is to make OTA available for Flatfish, but it takes some time and we should work on it step by step.
As I know, Danny has some plans on this and maybe he can provide some further advices.

@Danny, would you mind to share your opinions about making complete build with boot.img/recovery.img and also OTA/FOTA? Thanks.
Flags: needinfo?(jeliu) → needinfo?(dliang)
For generate boot.img and recovery.img, we have them in early stage but remove them due to some concerns: 
1. Device could be stuck if user flash abnormal boot.img
2. FOTA/OTA function could be failure if user flash abnormal recovery.img

For FOTA/OTA update, it should work but there is no conclusion about where to store FOTA/OTA image.

It's easy to meet build break because flatfish used 4.6 toolcahin, JB 4.2 and disable ril. I agreed to put flatfish build in try-server to avoid build break. For add build into try-server, we have some discussion in early stage, but it is suspended due to low priority.
Flags: needinfo?(dliang)
(In reply to Danny Liang [:dliang] from comment #12)
> For generate boot.img and recovery.img, we have them in early stage but
> remove them due to some concerns: 

Danny, I have succesfully build a working boot.img/recovery.img (actually didn't test the recovery.img yet). I've even integrated the kernel building into the build system as well, so my plan right now is make pull-request to the vendor repository in github to integrate all this work. I hope to do it this weekend.

> 1. Device could be stuck if user flash abnormal boot.img

Yeah, I "bricked" some of them in the past :), but with the new bootloader this could be easy recovered.

> 2. FOTA/OTA function could be failure if user flash abnormal recovery.img

Of course, but the goal of this bug is to provide a working recovery.img.
 
> For FOTA/OTA update, it should work but there is no conclusion about where
> to store FOTA/OTA image.

Because licenses?

> 
> It's easy to meet build break because flatfish used 4.6 toolcahin, JB 4.2
> and disable ril. I agreed to put flatfish build in try-server to avoid build
> break. For add build into try-server, we have some discussion in early
> stage, but it is suspended due to low priority.

I'm just about to close a bug related to the upgrade of the toolchain (bug 1056337), as soon as I close this bug the toolchain for Flatfish will be upgraded too (just for building Gecko, not Gonk).
Anyway, most of the build bustages are related to unsupported APIs because we are on ANDROID_VERSION 17 and JB4.3 (better supported) has 18. So yes, TCP program would LOVE to have a try-server build with our version.
Summary: FlatFish: Integrate boot.img and recovery.img into the build system → FlatFish: Integrate boot.img and kernel compilation into the build system
I couldn't make recovery image works so far. I'll focus on boot.img creation and kernel compilation from sources.
I'll suggest to keep a fork from the kernel sources in mozilla-b2g github account, so we can point to this fork in the manifest file.
As soon as we have this fork, I'll PR the vendor with all the changes needed.
Ok, I upload the kernel sources to github with the patches from the vendor, some other mine to compile with the google toolchain and integrate into the build system. The URL is: https://github.com/atilag/flatfish-kernel/tree/flatfish-b2g (I created a branch called flatfish-b2g).
Finally, we don't need to PR any vendor repository. We just need to merge with https://github.com/mozilla-b2g/device-flatfish.
To make things safer, I propose to create a branch in device-flatfish repo called something like: master-fullimage, so I can merge with it and test things with the try-server to ensure that nothing gets broken with these changes. This way, we have two options: compile the images as we are doing right now using the master branch, or compile all the images (except recovery.img) by changing to the new branch master-fullimage.
Flags: needinfo?(mwu)
Discussed on IRC yesterday. https://github.com/mozilla-b2g/kernel_flatfish was created. Now waiting for PRs to integrate all this.
Flags: needinfo?(mwu)
Attachment #8513035 - Flags: review?(mwu)
Attachment #8513032 - Flags: review?(mwu)
Attachment #8513032 - Flags: review?(mwu) → review+
What's nand.ko for? We build the kernel now, so we shouldn't need to include a binary kernel module, right?
It looks like the vendor uses it's own implementation of nand controller, and they provide a binary library: modules/nand/libnand, with no sources (GPL violation?.. someone claimed it [1]) and build the kernel module linking against this library.
This module is loaded by the init.rc script, and as I couldn't build it into the kernel, I left this untouched.

[1] http://linux-sunxi.org/GPL_Violations#NAND_support
Attachment #8513035 - Flags: review?(mwu)
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WONTFIX
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: