Closed Bug 840170 Opened 9 years ago Closed 9 years ago

b2g (on "master") fails to build for pandaboard with fresh gonk snapshot

Categories

(Firefox OS Graveyard :: GonkIntegration, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: rail, Assigned: jhford)

References

Details

Attachments

(3 files)

Attached file b2g-snapshot.sh
In bug 837019 I tried to update the gonk snapshot for mozilla-central. ATM panda fails to build with the following error:

07:04:52    ERROR -  make: *** No rule to make target `out/target/product/panda/obj/lib/libexif.so', needed by `out/target/product/panda/obj/SHARED_LIBRARIES/camera.omap4_intermediates/LINKED/camera.omap4.so'.  Stop.

The corresponding m-c try changeset is https://hg.mozilla.org/try/rev/c27a3210771b
and tbpl is https://tbpl.mozilla.org/?tree=Try&rev=c27a3210771b.

The snapshot was created with BRANCH=master exported. I manually checked the status of B2G/.repo/manifests directory:

$ git branch -va
* default                 4b8f040 Merge pull request #47 from ting-yuan/prelink
  remotes/m/master        -> origin/master
  remotes/origin/master   4b8f040 Merge pull request #47 from ting-yuan/prelink
  remotes/origin/nightly  20e8dc1 Merge pull request #38 from armenzg/nightly
  remotes/origin/v1-train f53e105 freezing revsion attributes for the v1-train branch
  remotes/origin/v1.0.0   272a722 freezing revsion attributes for the v1.0.0 branch

jhford tried to reproduce this, but it worked fine to him (I don't have the details though).

Could anyone help debugging the problem? Attached is the full script we use to generate snapshots. It was ran as "export BRANCH=master && ./b2g-snapshot.sh pandaboard"
jhford, could you describe the steps you followed to build the snapshot?
Flags: needinfo?(jhford)
(In reply to Rail Aliiev [:rail] from comment #1)
> jhford, could you describe the steps you followed to build the snapshot?

I didn't build a snapshot, rather I pulled down sources that would go into the snapshot and those weren't able to reproduce your error.
Flags: needinfo?(jhford)
Attached file releng-excludes
(In reply to John Ford [:jhford] from comment #2)
> (In reply to Rail Aliiev [:rail] from comment #1)
> > jhford, could you describe the steps you followed to build the snapshot?
> 
> I didn't build a snapshot, rather I pulled down sources that would go into
> the snapshot and those weren't able to reproduce your error.

jhford: 
1) Can you please try using the attached b2g-snapshot.sh to generate a snapshot on your machine, and see if that reproduces the build failure? Or if you prefer, Rail could send you a copy of the snapshot he generated. 
2) Also, it would be helpful if you can send us exact steps you ran, and output log, so we can verify if you and rail are actually attempting to build with the same bits...
(In reply to John O'Duinn [:joduinn] from comment #4)
> jhford: 
> 1) Can you please try using the attached b2g-snapshot.sh to generate a
> snapshot on your machine, and see if that reproduces the build failure? Or
> if you prefer, Rail could send you a copy of the snapshot he generated.

These snapshots are not a supported build configuration.  Rail, please use the supported build procedure to see whether or not this issue is limited in scope to the snapshot system or if it's a real code issue.

The simple build can be done with:

git clone git://github.com/mozilla-b2g/B2G.git &&
cd B2G &&
BRANCH=master ./config.sh pandaboard &&
./build.sh

> 2) Also, it would be helpful if you can send us exact steps you ran, and
> output log, so we can verify if you and rail are actually attempting to
> build with the same bits...

I don't have that information and I don't normally keep input and output logs for most things.
(In reply to John Ford [:jhford] from comment #5)
> These snapshots are not a supported build configuration.

Could you elaborate on this? Do you mean that the way how we build b2g in releng infra is not supported? I'm a little bit confused with the "these snapshots" phrase...
per today's b2g meeting: 

jhford and rail will meet tomorrow (wed, as soon as the v1.0.1 branching is completed) to debug. Vidyo would be most efficient, as the back/forth here already shows this topic is too tricky to debug by irc/bugmail.
Assignee: nobody → jhford
btw, the script perfectly worked for v1.0.1 manifests: https://tbpl.mozilla.org/?tree=Try&rev=39d33cb8ac6c
v1.0.0 worked fine as well https://tbpl.mozilla.org/?tree=Try&rev=1f070a0c5a23
The updated v1-train snapshot worked fine as well: https://bugzilla.mozilla.org/show_bug.cgi?id=840863
Rail and I just met.

Rail will try doing a build on a slave using the supported, standard build config to rule out the possibility of the snapshotting process causing the error.  Rail is also going to try without using the download-panda archive to test that it is using the correct downloaded file.

I am going to try figure out where libexif.so comes from and figure out which module installs it into the target build root.

Rail,  you can shorten your test to "./build.sh camera.omap4", which tells the build system to only build the camera.omap4 module, which is where the dependency failure you're seeing.

To prevent this issue from recurring, we should really have integration testing for Gonk changes on a regular basis
(In reply to Rail Aliiev [:rail] from comment #9)
> v1.0.0 worked fine as well
> https://tbpl.mozilla.org/?tree=Try&rev=1f070a0c5a23

(In reply to Rail Aliiev [:rail] from comment #8)
> btw, the script perfectly worked for v1.0.1 manifests:
> https://tbpl.mozilla.org/?tree=Try&rev=39d33cb8ac6c

(In reply to Rail Aliiev [:rail] from comment #10)
> The updated v1-train snapshot worked fine as well:
> https://bugzilla.mozilla.org/show_bug.cgi?id=840863

All other branches working with refreshed snapshots; only "master" breaks, so modifying summary to match.
Summary: b2g fails to build with fresh gonk snapshot → b2g (on "master") fails to build with fresh gonk snapshot
Rail, have you been able to reproduce using the supported build process?
Summary: b2g (on "master") fails to build with fresh gonk snapshot → b2g (on "master") fails to build for pandaboard with fresh gonk snapshot
Depends on: 841593
I found that the dependency that libexif.so comes from was removed from the manifests two weeks ago.  I have submitted a patch which adds jhead to the manifest.

Rail, please try to reproduce using:

curl -L "https://bugzilla.mozilla.org/attachment.cgi?id=714154" > manifest.xml &&
./config.sh pandaboard manifest.xml &&
rm -rf out/ &&
./build.sh -j1 camera.omap4

If that works, please run ./build.sh to see if you have any other errors.
Flags: needinfo?(rail)
(In reply to John Ford [:jhford] from comment #15)
> I found that the dependency that libexif.so comes from was removed from the
> manifests two weeks ago.  I have submitted a patch which adds jhead to the
> manifest.

This patch landed, please use the master branch.
I have just pushed https://tbpl.mozilla.org/?tree=Try&rev=58d29af4e127

If it fails, we can loan you a builder so you can debug the issue. I think, it will be just faster than if someone not that familiar with b2g build system would try to figure out what's the problem.
Flags: needinfo?(rail)
Panda now fails with a different error: see https://tbpl.mozilla.org/php/getParsedLog.php?id=19785832&tree=Try

15:01:38 ERROR - prebuilt/linux-x86/toolchain/arm-linux-androideabi-4.4.x/bin/../lib/gcc/arm-linux-androideabi/4.4.3/../../../../arm-linux-androideabi/bin/ld: out/target/product/panda/obj/EXECUTABLES/sutagent_intermediates/src/Logger.o: in function Logger::Logger():external/negatus/src/Logger.cpp:18: error: undefined reference to 'Config::instance()'
15:01:38 ERROR - prebuilt/linux-x86/toolchain/arm-linux-androideabi-4.4.x/bin/../lib/gcc/arm-linux-androideabi/4.4.3/../../../../arm-linux-androideabi/bin/ld: out/target/product/panda/obj/EXECUTABLES/sutagent_intermediates/src/SUTAgent.o: in function main:external/negatus/src/SUTAgent.cpp:184: error: undefined reference to 'Config::instance()'
15:01:38 ERROR - prebuilt/linux-x86/toolchain/arm-linux-androideabi-4.4.x/bin/../lib/gcc/arm-linux-androideabi/4.4.3/../../../../arm-linux-androideabi/bin/ld: out/target/product/panda/obj/EXECUTABLES/sutagent_intermediates/src/SUTAgent.o: in function main:external/negatus/src/SUTAgent.cpp:184: error: undefined reference to 'Config::setTestRoot(std::basic_string<char, std::char_traits<char>, std::allocator<char> >)'
15:01:38 ERROR - prebuilt/linux-x86/toolchain/arm-linux-androideabi-4.4.x/bin/../lib/gcc/arm-linux-androideabi/4.4.3/../../../../arm-linux-androideabi/bin/ld: out/target/product/panda/obj/EXECUTABLES/sutagent_intermediates/src/CommandEventHandler.o: in function CommandEventHandler::testroot(std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&):external/negatus/src/CommandEventHandler.cpp:851: error: undefined reference to 'Config::instance()'
15:01:38 ERROR - prebuilt/linux-x86/toolchain/arm-linux-androideabi-4.4.x/bin/../lib/gcc/arm-linux-androideabi/4.4.3/../../../../arm-linux-androideabi/bin/ld: out/target/product/panda/obj/EXECUTABLES/sutagent_intermediates/src/CommandEventHandler.o: in function CommandEventHandler::rebt(std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&):external/negatus/src/CommandEventHandler.cpp:675: error: undefined reference to 'Config::instance()'
15:02:05 ERROR - Return code: 2
15:02:05 FATAL - failed to build
15:02:05 FATAL - Exiting 2
Any ideas?
(In reply to Rail Aliiev [:rail] from comment #19)
> Any ideas?

Looks like it's a negatus (sut agent) build failure.  The A-Team knows this code best.  Try doing rm -rf $B2G_ROOT/external/negatus right before ./build.sh (or use a manifest that doesn't have negatus) and see if it compiles.  If it does, then the problem is limited to negatus.

To isolate to building just negatus, you can run "./build.sh -j1 sutagent".
Depends on: 843221
per mtg w/jhford:

1)negatus issues look resolved in bug#843221. No manifest updates needed.

2) rail, please try new builds on m-c and see if we hit further problems.
This worked fine with changes from bug 843221
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.