Closed Bug 1039739 Opened 6 years ago Closed 6 years ago

Flame builds no longer have gaia commit info in settings app

Categories

(Firefox OS Graveyard :: Gaia::Build, defect)

x86_64
Linux
defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: wlach, Assigned: yurenju)

References

Details

Attachments

(3 files)

We used to have a file in the setting's application's 'application.zip' in resources/ called gaia_commit.txt which had gaia commit info. This seems to no longer be present in builds, which breaks getting proper gaia version info for eideticker and probably other testing tools via mozversion (http://hg.mozilla.org/mozilla-central/file/f6e46d1fc903/testing/mozbase/mozversion/mozversion/mozversion.py#l123).

I did some quick bisection on b2g-inbound on flame, and it looks like this problem first starts occuring in this build:

https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-builds/b2g-inbound-flame-eng/20140716013201/

The previous inbound build doesn't have this problem:

https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-builds/b2g-inbound-flame-eng/20140716012802/

The commit on the inbound build with the problems is d74dd8f351866264f198985a9414633e957dbfff, which corresponds to:

https://github.com/mozilla-b2g/gaia/commit/d74dd8f351866264f198985a9414633e957dbfff

This commit was intended to fix creating this file on windows, but it also alters the way we copy the git commit override file into the settings application. I *believe* (not 100% sure) we use a commit file override in the gaia build process, so perhaps this change isn't working as expected:

https://github.com/mozilla-b2g/gaia/commit/d74dd8f351866264f198985a9414633e957dbfff#diff-00aa0db2ae450f56abde226f0551699bL91

Yuren, did you test this codepath before committing? It doesn't actually seem to have anything to do with your intended change, if I understand it correctly. Maybe the solution is to just back that one part out of the patch?
Flags: needinfo?(yurenju.mozilla)
:wlach, I use the command to verify but looks good:

$ echo "aaa" > gaia_commit_override.txt && APP=settings make && cat build_stage/settings/resources/gaia_commit.txt

I'll investigate it.
I made a quick patch to restore that part to override commit file in settings/build/build.js but I'm not sure it can fix the problem since it can not be reproduced on my side.

:wlach, do you have any way to test this pull request if it can fix that problem?
Flags: needinfo?(yurenju.mozilla) → needinfo?(wlachance)
See Also: → 1040069
This works for me too. Aki, do you have any idea why this change might cause the gaia revision to not be picked up by the build?
Flags: needinfo?(wlachance) → needinfo?(aki)
I don't, no.  Usually when I'm faced with a problem like this I add more verbose logging around the problem.
Flags: needinfo?(aki)
One thing that looks potentially suspicious is this line:

  git.initPath(utils.getEnvPath());

I'm not sure what this does, but it may be relying on something that's not set up on the slaves.
(In reply to Jonathan Griffin (:jgriffin) from comment #5)
> One thing that looks potentially suspicious is this line:
> 
>   git.initPath(utils.getEnvPath());
> 
> I'm not sure what this does, but it may be relying on something that's not
> set up on the slaves.

This was set up before without causing any problems:

https://github.com/mozilla-b2g/gaia/commit/d74dd8f351866264f198985a9414633e957dbfff#diff-00aa0db2ae450f56abde226f0551699bL96

I did some detective work, and it looks like, in fact, we *are* using git to get the revision in the builds, I see this kind of log message in the build:

[cmd] git --git-dir=/builds/slave/b2g_b2g-in_flame_eng_dep-00000/build/gaia/.git log -1 --format=%H%n%ct HEAD

HOWEVER, I found out that we're not emitting any error messages if the git command doesn't succeed, which we should be. I'll file a seperate bug for this.

The only thing I can think of for the problem is that we were quoting the string in --format before, whereas we aren't now. I'm not sure though. Almost certainly the command is erroring out somehow though. I suggest we get in some code to add some logging to figure out what's going on, then do up a proper fix.
Attachment #8458198 - Flags: review?(yurenju.mozilla)
Comment on attachment 8458198 [details] [review]
Pull request to add logging

r=yurenju and your pr have some jshint error, I will fork your pr and fix it before merge.
Attachment #8458198 - Flags: review?(yurenju.mozilla) → review+
Thanks Yuren! So we're now getting a bit of debugging information:

E.g. https://tbpl.mozilla.org/php/getParsedLog.php?id=44126836&tree=B2g-Inbound&full=1 :

10:56:09     INFO -  [cmd] git --git-dir=/builds/slave/b2g_b2g-in_flame_eng_dep-00000/build/gaia/.git log -1 --format=%H%n%ct HEAD
10:56:09     INFO -  [settings-app-build] Error writing git commit file!
10:56:09     INFO -  stderr:
10:56:09     INFO -  undefined
10:56:09     INFO -  stdout: undefined

I did a bit of investigation and don't see why this wouldn't work. It's a very mysterious error. At least locally, if I try to run a command that we can't find (i.e. git wasn't in $PATH), I get a very obvious error. Likewise if I pass arguments to git that it doesn't understand. So I'm not sure what's going on. I think the best thing to do is to figure out the exact location of the git binary that we're trying to execute. I think that might give us a better hint about exactly what's happening here. I'll do up another pull request to get that information.

Also, I think we can say that we should not continue with the build if we get an error like this, in general. Filed bug 1040918 for that. :)
Comment on attachment 8458888 [details] [review]
Make it more clear what command we're actually executing (e.g. /usr/bin/git vs. just git)

r=yurenju,

https://github.com/mozilla-b2g/gaia/commit/cab9e0dd0dee93f31f12313d012e660f074663cf
Attachment #8458888 - Flags: review?(yurenju.mozilla) → review+
try to clone B2G and build for flame but I got gaia_commit.txt in gaia/profile/settings.gaiamobile.org/application.zip.
Request a loaner machine for this issue on bug 1039739.
Yuren, what is the ETA on this? It's affected our reporting and a bit of a nuisance. We can wait a day or two but if it's in the region of a week or more can we back out and re-land it with the fix?
We just ran a build with my patch applied. Here's what we're seeing on the Flame:

01:04:32     INFO -  [cmd] /usr/local/bin/git --git-dir=/builds/slave/b2g_b2g-in_flame_eng_dep-00000/build/gaia/.git log -1 --format=%H%n%ct HEAD
01:04:32     INFO -  [settings-app-build] Error writing git commit file!
01:04:32     INFO -  stderr:
01:04:32     INFO -  undefined
01:04:32     INFO -  stdout: undefined

I just did some testing and can replicate this error message from the gaia build in the following scenarios:

1. User does not have permission to access /usr/local/bin/git
2. /usr/local/bin/git is an empty file or otherwise non-executable

(git errors always produce at least some output)

On the Hamachi, we're seeing the same result with /usr/bin/git.

:aki, does this give any clues as to why this might be happening? Could you see any reason why the gaia build system might not have permission to access git?
Flags: needinfo?(aki)
I think we can reproduce it on loaner machine, so let's back out commit on bug 1036821
backed out three commits:

[master cb854d0] Revert "Bug 1039739 - Make it more clear what command we're executing;r=yuren"

[master 00349ce] Revert "Bug 1039739 - Add some logging when writing the git commit id into the settings app;r=yurenju"

[master e92c1ce] Revert "Bug 1036821 - git commit hash isn't wrote to settings/resources/gaia_commit.txt on Windows"

:wlach, could you mark it as reolved if those revert commits can rosolve the problem?
Flags: needinfo?(wlachance)
(In reply to Yuren [:yurenju] from comment #18)
> backed out three commits:
> 
> [master cb854d0] Revert "Bug 1039739 - Make it more clear what command we're
> executing;r=yuren"
> 
> [master 00349ce] Revert "Bug 1039739 - Add some logging when writing the git
> commit id into the settings app;r=yurenju"
> 
> [master e92c1ce] Revert "Bug 1036821 - git commit hash isn't wrote to
> settings/resources/gaia_commit.txt on Windows"
> 
> :wlach, could you mark it as reolved if those revert commits can rosolve the
> problem?

Yes, this fixes the problem, I see the commit file in the latest versions of gaia. However, I don't think we should back out my changes as they provide information which might be useful in debugging either this or similar problems in the future and are almost certainly not the cause of the bug (since they were applied after we recognized bug 1036821 as the cause). Could you reapply them? I'm fine with resolving this and following up in bug 1036821 after that.
Flags: needinfo?(wlachance) → needinfo?(yurenju.mozilla)
Let's follow up with aki's needinfo request in the original bug.
Flags: needinfo?(aki)
Wlach, sure, I backed them out because your commits are top of my commit so back all of three commits out  is easy then extract my commit. I will re-land it if it can be applied to current settings/build/build.js

BTW, I'm leaving office, so feel free to apply it or I will do it tomorrow.
Flags: needinfo?(yurenju.mozilla)
This issue is fixed in the latest master build. The commit info is displayed in Settings app. 

Build info:
Gaia      cd50cb6ddeb430ddaf43e4fcf5774b9d262a8cf3
Gecko     https://hg.mozilla.org/mozilla-central/rev/e5ced39f443b
BuildID   20140722160202
Version   34.0a1
ro.build.version.incremental=109
ro.build.date=Mon Jun 16 16:51:29 CST 2014
mark as resolved since we have backout commit from bug 1036821.
Assignee: nobody → yurenju.mozilla
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.