1020259 - Allow generating reproducible gaia builds

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::Build, defect)

Description

[Alexandre Poirot [:ochameau]]

Attachment: Pull request 20000

For now, gaia build system has a non deterministic output.
Mostly due to external hosted apps that end up being store in folders with random names.
But we also have install/update time fields in webapps.json that is set to a less random, but still unpredictable value.
And finally, in settings, the git revision and date, which is predictable but it would be handy to control it.

This non deterministic output introduce an unexpected behavior: if you build gaia out of the same git revision, but, clean stage/profile between each run, you will have a different output! Different time fields, different ids for external hosted apps.

I'm trying to get rid of this behavior in order to ease comparing two stage or profile folders from two different gaia revision. That to see if a changeset change anything to the build system output.

With this patch, that introduces yet another flag "REPRODUCIBLE", you can do the following:
$ git checkout upstream/master
$ REPRODUCIBLE=1 make
$ mv profile master_profile && mv build_stage master_stage
$ make clean
$ git checkout dev/patch
$ REPRODUCIBLE=1 make
$ diff -r master_stage build_stage
$ diff -r master_profile profile

(such practice allows me to better review build system patches, like bug 1017490)

And the two diff results are going to be empty!

I'm not completely confident about this patch:
* We may not want to introduce yet another variable and just make the build more deterministic
* I don't know why bug 929602 started using random name for app folders, but that's really something we should stop doing in order to fix `make install-gaia`. (if you happen to clean stage/profile, make install-gaia will mess up webapps.json and push new duplicated external apps)
* The git revision/date isn't about being deterministic, it is really more related to my usecase of doing diff, may be we should just come up with a special diff script that would allow ignoring this particular difference in settings.
* Regarding install/update time... is it actually being useful/used at all?

Comment 1

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Comment on attachment 8434078 [details] [review]
Pull request 20000

the reason for generating uuid directories name, :amac explained it on bug 927967 comment 12, and another issue for update/install time, we change it from fixed time to get time from |Date.now()| on bug 989876.

but it is still worth to have this feature since I also have same problem when I was debugging/verifing output of build system.

for replacement of uuid, I suggest using webapp.sourceDirectoryFile.leafName.

Comment 2

[BMO Automation]

Firefox OS is not being worked on