Move the Firefox OS update documentation to MDN and update it with the new FOTA targets

RESOLVED FIXED

Status

RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: gsvelto, Assigned: cmills)

Tracking

Details

(URL)

(Reporter)

Description

5 years ago
Now that bug 935059 has landed it would be nice to move the documentation that is currently at https://wiki.mozilla.org/B2G/Updating into MDN and to update it to cover the new build targets used to generate full and partial FOTA updates.
This sounds fine. The document mostly makes sense, and I can always throw questions at you as they come up. 

Do you have a particular deadline in mind for this? I ask because I am busy at work weeks for the next 2 weeks, and I am also currently working on a structural reorganization of the MDN Firefox OS docs zone, as it is could be better organized.

If this is urgent, then I will get it transferred over to MDN tomorrow, incrementally improve on it on it, and move it if needed during the organization.

If it is not so urgent, I'll get my work weeks and the FxOS reorg out of the way first, and then transfer this doc over.

What would you prefer?
Status: NEW → ASSIGNED
(Reporter)

Comment 2

5 years ago
(In reply to Chris Mills from comment #1)
> This sounds fine. The document mostly makes sense, and I can always throw
> questions at you as they come up.

Sounds great! BTW I'm changing the bug title, I forgot to do it when I cloned the original bug.

> If it is not so urgent, I'll get my work weeks and the FxOS reorg out of the
> way first, and then transfer this doc over.
> 
> What would you prefer?

Take your time, there's no pressure at all. Like bug 939415 (for which I hope to provide you the information ASAP) this is all pretty low-level documentation mostly of interest to hackers and possibly our partners; most of the FxOS developers won't ever need it.
Summary: Implement a FOTA update solution for Buri devices → Move the Firefox OS update documentation to MDN and update it with the new FOTA targets
FYI, this is valid for not only buri, but it should work on all devices ; at least when I tested I successfully used: keon, peak, nexus s, nexus 4 and inari. Some might be dependent on bug 952535.
I have added the document to MDN:

https://developer.mozilla.org/en-US/Firefox_OS/Building_and_installing_Firefox_OS/Creating_Firefox_OS_update_packages

I also had some questions:

1. What does "target files zip for the device partitions" mean?

2. Isn't something missing from the "Generating a complete Gecko/Gaia OTA update MAR" and Generating a complete FOTA update zip and target files zip" sections? Don't you have to generate these items from the build you are updating from too? i.e., version X? I know this is probably implicit, but I think we should probably spell all steps out explicitly.

3. In the "Hosting updates and polling for updates on the client side", where you show the example URL, it would be nice to have an explicit explanation of what parameters correspond to which parts of the URL. If you could first show a URL with parameters in it, then the table, then the real example, this would be good.

4. In the manifest example, you should say explicitly which bit of XML corresponds to which field described below.

5. "The details of the interaction between build servers and the update host is currently beyond the scope of this document. It is highly dependent on the production environment." - where do developers get this information from then? This seems like a glaring omission.

6. "The first step in applying an update is to verify the signatures embedded in the MAR packages." - so how is this done? Nowhere is the process of adding the signatures to the packages explained. Is this in the out of scope bit? 

7. In the "Verifying and applying updates" section, it seems to me like it would be more complete if we wrote a couple of paragraphs on how the user of the device actually applies the update. Pointing their browser at the package, downloading it to the computer and then Flashing, or putting it on an SDCard, etc?
These docs would be handy but they are a bit broken at the moment. I think cmills questions above would address most of my current concerns as well. Is there any progress being made here?
Flags: needinfo?(cmills)
Flags: needinfo?(gsvelto)
Under section "Generating a complete FOTA update zip and target files zip" these instructions are provided to generate the target files zip:

$ cd $b2g
$ ./build.sh

While this supposedly works for hamachi, it does not for neither peak or dolphin.

Instead you have to do (and this should always work, also for hamachi):

$ cd $b2g
$ ./build.sh target-files-package

It would be great if this was reflected in the docs as well.
(In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #6)
> Under section "Generating a complete FOTA update zip and target files zip"
> these instructions are provided to generate the target files zip:
> 
> $ cd $b2g
> $ ./build.sh
> 
> While this supposedly works for hamachi, it does not for neither peak or
> dolphin.
> 
> Instead you have to do (and this should always work, also for hamachi):
> 
> $ cd $b2g
> $ ./build.sh target-files-package
> 
> It would be great if this was reflected in the docs as well.

Updated:

https://developer.mozilla.org/en-US/Firefox_OS/Building_and_installing_Firefox_OS/Firefox_OS_update_packages#Generating_a_complete_FOTA_update_zip_and_target_files_zip_2
I'm cancelling my needinfo request here, as I've updated the requested section, and it is really Gabriele's input we need on this one now. I'll see if I can follow up on mail.
Flags: needinfo?(cmills)
(Reporter)

Comment 9

4 years ago
(In reply to Chris Mills (Mozilla, MDN editor) [:cmills] from comment #4)
> 1. What does "target files zip for the device partitions" mean?

It's the file referenced in https://developer.mozilla.org/en-US/Firefox_OS/Building_and_installing_Firefox_OS/Firefox_OS_update_packages#Generating_a_complete_FOTA_update_zip_and_target_files_zip_2 which is called $DEVICE-target_files-$VARIANT.$USER.zip

This file is a ZIP holding all the files for the various partitions on the phone. Every partition is a root directory with the name written in all caps, e.g.:

SYSTEM/
BOOT/
RECOVERY/
DATA/
etc...

The SYSTEM/ folder for example will contain all the files that will go in the /system mountpoint (and that are also stored in the system.img partition image when doing a regular build).

> 2. Isn't something missing from the "Generating a complete Gecko/Gaia OTA
> update MAR" and Generating a complete FOTA update zip and target files zip"
> sections? Don't you have to generate these items from the build you are
> updating from too? i.e., version X? I know this is probably implicit, but I
> think we should probably spell all steps out explicitly.

Yes, version X needs to be available somehow. The article mentions as an example that it can be taken from the build servers but that's obviously an example which makes sense for our own build. The most common scenario for an external vendor/contributor will be to build both X and Y versions on his machine and generate the update with them so clarifying this part is a good idea.

> 3. In the "Hosting updates and polling for updates on the client side",
> where you show the example URL, it would be nice to have an explicit
> explanation of what parameters correspond to which parts of the URL. If you
> could first show a URL with parameters in it, then the table, then the real
> example, this would be good.

Good point, in the given example that would be:

https://updates.b2g.com/$CHANNEL/$PRODUCT_MODEL/$VERSION/$BUILD_ID/update.xml

Pardon the bash-iness of my example :)

> 4. In the manifest example, you should say explicitly which bit of XML
> corresponds to which field described below.

There's no direct correspondence, the update.xml files follow the schema we use for Firefox updates too and which is described here:

https://wiki.mozilla.org/Software_Update:updates.xml_Format

I think we're missing certain bits however, for example bug 1000221 introduced an "isOSUpdate" attribute to distinguish between FOTA and OTA updates.

> 5. "The details of the interaction between build servers and the update host
> is currently beyond the scope of this document. It is highly dependent on
> the production environment." - where do developers get this information from
> then? This seems like a glaring omission.

That's a question best asked to the release engineering people. The documentation for that process should be the same as for regular Firefox updates so https://wiki.mozilla.org/Software_Update but something clearer and more tailored to Firefox OS would help.

> 6. "The first step in applying an update is to verify the signatures
> embedded in the MAR packages." - so how is this done? Nowhere is the process
> of adding the signatures to the packages explained. Is this in the out of
> scope bit?

I'm not very familiar with MAR signatures but the text in https://developer.mozilla.org/en-US/Firefox_OS/Building_and_installing_Firefox_OS/Firefox_OS_update_packages#Generating_an_incremental_FOTA_update_zip_2 indicates that the build/tools/releasetools/ota_from_target_file script already signs the MAR file when provided with a key via the -k option.

> 7. In the "Verifying and applying updates" section, it seems to me like it
> would be more complete if we wrote a couple of paragraphs on how the user of
> the device actually applies the update. Pointing their browser at the
> package, downloading it to the computer and then Flashing, or putting it on
> an SDCard, etc?

This is currently handled from within Firefox OS. This is detailed here:

https://support.mozilla.org/en-US/kb/how-do-i-check-firefox-os-updates-and-install-them

It is possible to also apply a FOTA update manually by it's not very common. The process is described in bug 935059 comment 35. This usually done only for development.
Flags: needinfo?(gsvelto)
I've been digging through both full and incremental ota and fota updates lately so I'll chip in a bit here.

(In reply to Gabriele Svelto [:gsvelto] from comment #9)
> (In reply to Chris Mills (Mozilla, MDN editor) [:cmills] from comment #4)
> > 3. In the "Hosting updates and polling for updates on the client side",
> > where you show the example URL, it would be nice to have an explicit
> > explanation of what parameters correspond to which parts of the URL. If you
> > could first show a URL with parameters in it, then the table, then the real
> > example, this would be good.
> 
> Good point, in the given example that would be:
> 
> https://updates.b2g.com/$CHANNEL/$PRODUCT_MODEL/$VERSION/$BUILD_ID/update.xml
> 
> Pardon the bash-iness of my example :)
I have been looking for the %variables% you can put into the url but not found an authoritative source for them. I think it would be great to have that in the docs. The default app.update.url is now set to (in gecko/b2g/app/b2g.js) "https://aus4.mozilla.org/update/3/%PRODUCT%/%VERSION%/%BUILD_ID%/%PRODUCT_DEVICE%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml", and contains a lot more variables than in the example. I'd like to know all that are available and what variables from the build they resolve to.

> > 4. In the manifest example, you should say explicitly which bit of XML
> > corresponds to which field described below.
> 
> There's no direct correspondence, the update.xml files follow the schema we
> use for Firefox updates too and which is described here:
> 
> https://wiki.mozilla.org/Software_Update:updates.xml_Format
> 
> I think we're missing certain bits however, for example bug 1000221
> introduced an "isOSUpdate" attribute to distinguish between FOTA and OTA
> updates.
Here it would be great to mention the script at B2G/tools/update-tools/build-update-xml.py. It is quite self-explanatory although some terms should probably be explained. Like partial and complete updates, APP_VERSION and PLATFORM_VERSION (both are the gecko version?), and where the actual build id can be found (out/target/product/$DEVICE/system/b2g/application.ini).

> > 6. "The first step in applying an update is to verify the signatures
> > embedded in the MAR packages." - so how is this done? Nowhere is the process
> > of adding the signatures to the packages explained. Is this in the out of
> > scope bit?
> 
> I'm not very familiar with MAR signatures but the text in
> https://developer.mozilla.org/en-US/Firefox_OS/
> Building_and_installing_Firefox_OS/
> Firefox_OS_update_packages#Generating_an_incremental_FOTA_update_zip_2
> indicates that the build/tools/releasetools/ota_from_target_file script
> already signs the MAR file when provided with a key via the -k option.
Note that it is not the MAR file that gets signed, it's the fota zip file that gets bundled into the MAR that gets signed by build/tools/releasetools/ota_from_target_file. (The script doesn't have anything to do with MAR files in fact)
The signing of the fota update is total android style and I think that should be mentioned. If you just run the script without specifying the key, it will use the developer key at build/target/product/security/testkeys.*. Fine for testing but when a vendor creates an update they'd want a secure key, i.e., one that no-one else knows about. The device will also verify that signature before applying the patch, so a device's initial images will need to contain the key as well. I haven't gone through with this so I don't yet know how to pass the key in when building the images but it should be covered in the docs if you ask me.
(In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #10)
> Note that it is not the MAR file that gets signed, it's the fota zip file
> that gets bundled into the MAR that gets signed by
> build/tools/releasetools/ota_from_target_file. (The script doesn't have
> anything to do with MAR files in fact)
> The signing of the fota update is total android style and I think that
> should be mentioned. If you just run the script without specifying the key,
> it will use the developer key at build/target/product/security/testkeys.*.
> Fine for testing but when a vendor creates an update they'd want a secure
> key, i.e., one that no-one else knows about. The device will also verify
> that signature before applying the patch, so a device's initial images will
> need to contain the key as well. I haven't gone through with this so I don't
> yet know how to pass the key in when building the images but it should be
> covered in the docs if you ask me.

Well, this applies to FOTA updates as mentioned under https://developer.mozilla.org/en-US/Firefox_OS/Building_and_installing_Firefox_OS/Firefox_OS_update_packages#FOTA_updates_2

Non-incremental FOTA as generated by build targets gecko-update-fota and gecko-update-fota-full seem to be created by the scripts in tools/update-tools instead of build/tools/releasetools/ota_from_target_Files. They also take keys and default to the same dev-keys as ota_from_target_files, so some kind of key management is definitely needed for a vendor making its own releases.

I didn't know the MAR files are actually signed. Nothing in the tools seem to mention this, so it would be nice to know how it works. E.g., which keys are used?
(Reporter)

Comment 12

4 years ago
(In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #10)
> I have been looking for the %variables% you can put into the url but not
> found an authoritative source for them. I think it would be great to have
> that in the docs. The default app.update.url is now set to (in
> gecko/b2g/app/b2g.js)
> "https://aus4.mozilla.org/update/3/%PRODUCT%/%VERSION%/%BUILD_ID%/
> %PRODUCT_DEVICE%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/
> %DISTRIBUTION_VERSION%/update.xml", and contains a lot more variables than
> in the example. I'd like to know all that are available and what variables
> from the build they resolve to.

That's a question for RelEng really, I think we inherit some of those variables from the Firefox update mechanism (e.g. %LOCALE% which shouldn't be important in Firefox OS since it's multi-locale). Anyway I don't think the URL we use for updates is relevant to a vendor; they can use any URL format they wish instead provided that the update.xml file contains the proper data.

> Here it would be great to mention the script at
> B2G/tools/update-tools/build-update-xml.py. It is quite self-explanatory
> although some terms should probably be explained. Like partial and complete
> updates,

IIRC they're both FOTA updates with a complete update overwriting all partitions and a partial one only the files under /system/b2g (and maybe something else like fonts IIRC).

> APP_VERSION and PLATFORM_VERSION (both are the gecko version?), and
> where the actual build id can be found
> (out/target/product/$DEVICE/system/b2g/application.ini).

I think they are the same but don't take my word for it.

> I haven't gone through with this so I don't
> yet know how to pass the key in when building the images but it should be
> covered in the docs if you ask me.

Yes indeed though it should work pretty much like on Android. As for the MAR signature (which is inherited from desktop Firefox) I'm not sure how that works unfortunately. I'm needinfo'ing release engineering in the hope they can help clarify some of your questions I couldn't answer.
Flags: needinfo?(release)
Flags: needinfo?(lissyx+mozillians)
I think that what you want is all explained at https://github.com/mozilla-b2g/B2G/blob/master/tools/update-tools/build-flash-fota.py#L96
Flags: needinfo?(lissyx+mozillians)
(In reply to Alexandre LISSY :gerard-majax from comment #13)
> I think that what you want is all explained at
> https://github.com/mozilla-b2g/B2G/blob/master/tools/update-tools/build-
> flash-fota.py#L96

That is just signing the fota .zip so I still don't know how a .mar gets signed.

(In reply to Gabriele Svelto [:gsvelto] from comment #12)
> IIRC they're both FOTA updates with a complete update overwriting all
> partitions and a partial one only the files under /system/b2g (and maybe
> something else like fonts IIRC).

Well then the build-update-xml.py script has this somewhat confusing part in its usage help :)
  -O, --fota-update     The complete MAR contains a FOTA update. Default:
                        detect. Note: only 'complete' MARs can be FOTA
                        updates.
Ok, I've done some more updates, to cover changes requested down to about comment 10. I am sure there are some unanswered questions though.

Also, you mention a couple of build tools/locations, e.g. 

build/target/product/security/testkeys.*
build/tools/releasetools/ota_from_target_file

Where are these actually located?
(Reporter)

Comment 16

4 years ago
(In reply to Chris Mills (Mozilla, MDN editor) [:cmills] from comment #15)
> build/target/product/security/testkeys.*
> build/tools/releasetools/ota_from_target_file

They're both found under the main B2G source repository, in the former case you're generally looking at these files:

<B2G sources path>/build/target/product/security/testkey.pk8
<B2G sources path>/build/target/product/security/testkey.x509.pem

And in the latter:

<B2G sources path>/build/tools/releasetools/ota_from_target_files

Note that it's ota_from_target_files not ota_from_target_file
Chris, I updated the updating page with some info on signing builds and FOTAs as I've spent a lot of time the last month going through these steps and fixing the tools. Feel free to take a look and review.

I could add how to generate ota updates and/or images from the target files zips, but since ota updates are already covered using a build target and images are not strictly needed for instructions on building updates, I chose to not include it.
(In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #17)
> Chris, I updated the updating page with some info on signing builds and
> FOTAs as I've spent a lot of time the last month going through these steps
> and fixing the tools. Feel free to take a look and review.
> 
> I could add how to generate ota updates and/or images from the target files
> zips, but since ota updates are already covered using a build target and
> images are not strictly needed for instructions on building updates, I chose
> to not include it.

Thanks Andreas - this looks cool to me.

Gabriele, can you give this a quick tech review as well? If you are happy, then I think we can close this bug. Thanks.
Flags: needinfo?(gsvelto)
(Reporter)

Comment 19

4 years ago
Shall I review this page?

https://developer.mozilla.org/en-US/Firefox_OS/Building_and_installing_Firefox_OS/Firefox_OS_update_packages

It doesn't have the technical review flag set so I'm not sure...
(In reply to Gabriele Svelto [:gsvelto] from comment #19)
> Shall I review this page?
> 
> https://developer.mozilla.org/en-US/Firefox_OS/
> Building_and_installing_Firefox_OS/Firefox_OS_update_packages
> 
> It doesn't have the technical review flag set so I'm not sure...

Apologies, I forgot to set it - yes please do!
(Reporter)

Comment 21

4 years ago
(In reply to Chris Mills (Mozilla, MDN editor) [:cmills] from comment #20)
> Apologies, I forgot to set it - yes please do!

Chris, I had set some time aside for doing the technical review this week but I see that the flag is not set on that page. Was the article already reviewed by someone else?
Flags: needinfo?(gsvelto)
(In reply to Gabriele Svelto [:gsvelto] from comment #21)
> (In reply to Chris Mills (Mozilla, MDN editor) [:cmills] from comment #20)
> > Apologies, I forgot to set it - yes please do!
> 
> Chris, I had set some time aside for doing the technical review this week
> but I see that the flag is not set on that page. Was the article already
> reviewed by someone else?

D'oh - I was sure I'd set that. It is now set. no one else has reviewed it, to my knowledge.
(Reporter)

Comment 23

4 years ago
Quick update, I'm going through the article and I've stumbled upon a bug in the scripts used to sign packages which prevent them from working with the Flame builds (duh). I'll proceed to fixing it before going on with the review. I've also noticed another thing: it seems that we're still missing information about partial FOTA updates. I'll file a bug for that and add it myself ASAP.
(Reporter)

Comment 24

4 years ago
I've finished the technical review of the article, sorry that it took so long.
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Flags: needinfo?(release)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.