1025788 - [Flame] build tcpdump

Status: RESOLVED FIXED

(Firefox OS Graveyard :: GonkIntegration, defect)

Description

[:gerard-majax]

As far as I can say, we do not build tcpdump for the Flame.

Adding:
> <project name="platform/external/tcpdump" path="external/tcpdump"/>

To the manifest should be enough.

Comment 1

[:gerard-majax]

We also need libpcap:

> <project name="platform/external/libpcap" path="external/libpcap"/>

That does produce a binary in xbin:
> $ find out/target/product/flame/system/ | grep tcpdump
> out/target/product/flame/system/xbin/tcpdump

Comment 2

[:gerard-majax]

Attachment: Manifest update

Michael, please find attached the pull request that adds the needed bits to the device manifest. I'm wondering about two points:
 - maybe we should do this against base-jb.xml and base-jb-caf.xml
 - we need to mirror those repos, right?

Comment 3

[Michael Wu [:mwu]]

(In reply to Alexandre LISSY :gerard-majax from comment #2)
> Created attachment 8443388 [details] [review]
> Manifest update
> 
> Michael, please find attached the pull request that adds the needed bits to
> the device manifest. I'm wondering about two points:
>  - maybe we should do this against base-jb.xml and base-jb-caf.xml

Yes, base-jb.xml sounds good.

>  - we need to mirror those repos, right?

Yup. (unless someone mirrored them already)

Comment 4

[:gerard-majax]

As far as I can say, Flame includes base-caf-jb, not base-jb.

Comment 5

[Michael Wu [:mwu]]

Include it in both.

In the kk manifests, base-kk-aosp includes base-kk, and base-kk-caf will include base-kk when it's introduced.

Comment 6

[Michael Wu [:mwu]]

Comment on attachment 8443388 [details] [review]
Manifest update

Per discussion, waiting for a PR that adds this to the applicable base manifests.

Comment 7

[:gerard-majax]

Comment on attachment 8443388 [details] [review]
Manifest update

Michael, new PR pushed with libpcap and tcpdump in:
 - base-caf-jb
 - base-caf
 - base-kk

Comment 8

[Michael Wu [:mwu]]

Comment on attachment 8443388 [details] [review]
Manifest update

Looks good. As usual, make sure there are mirrors before merging.

Comment 9

[:gerard-majax]

https://github.com/mozilla-b2g/b2g-manifest/commit/7031c46b9bd6f9ba1b83024576a02045fe169a7c

Comment 10

[Ed Morley [:emorley]]

(In reply to Alexandre LISSY :gerard-majax from comment #9)
> https://github.com/mozilla-b2g/b2g-manifest/commit/
> 7031c46b9bd6f9ba1b83024576a02045fe169a7c

Reverted on suspicion of causing bug 1031354.

https://github.com/mozilla-b2g/b2g-manifest/commit/de6d982e708fc2df69a1051330aab5063f89e00e

Comment 11

[:gerard-majax]

Can I get an explanation ? According to bug 1031378 it's the lack of mirroring of the new repos.
I did asked and waited for mirrors to be configured, in bug 1029342.

Comment 12

[:gerard-majax]

Okay the URLs from bug 1029342 were wrong.

Comment 13

[Ed Morley [:emorley]]

(In reply to Alexandre LISSY :gerard-majax from comment #11)
> Can I get an explanation ? According to bug 1031378 it's the lack of
> mirroring of the new repos.
> I did asked and waited for mirrors to be configured, in bug 1029342.

In cases like this, you need to ask the person assigned to that bug, or else someone else from releng, since the sheriffs do not set up or maintain repos on disk, mirroring to git.m.o or the b2g bumper bot.

Comment 14

[:gerard-majax]

Attachment: Manifest update, rebased

Carrying r+, rebasing the manifest on the current master after the backout. There is no emergency to land this, so I'll needinfo? myself to make sure we land it after the 2.0 hurry period.

Comment 15

[:gerard-majax]

Relanded, hopefully this will not break again :)
https://github.com/mozilla-b2g/b2g-manifest/commit/b5c6241c2e598e14ddfea126ee598c7e3c9430c9

Comment 16

[Pete Moore [:pmoore][:pete]]

I'll keep my eye out for problems on this

Comment 17

[Pete Moore [:pmoore][:pete]]

(long term solution: bug 1031378)

Comment 18

[:gerard-majax]

Okay for some reason this is not working as expected:

> 05:55:57  WARNING - Returned 128 - sleeping and retrying
> 05:55:57  WARNING - Got output: fatal: remote error: FATAL: R any external/caf/platform/external/libpcap gitweb DENIED by fallthru
> 05:55:57  WARNING - (or you mis-spelled the reponame)
> 05:55:59  WARNING - Returned 128 - sleeping and retrying
> 05:55:59  WARNING - Got output: fatal: remote error: FATAL: R any external/caf/platform/external/tcpdump gitweb DENIED by fallthru
> 05:55:59  WARNING - (or you mis-spelled the reponame)

Comment 19

[Pete Moore [:pmoore][:pete]]

B2G bumper script is here:
http://hg.mozilla.org/build/mozharness/file/375d8b60ae6b/scripts/b2g_bumper.py

The answer will be in here.

The docs probably need to be improved on this topic - I'll investigate.

Comment 20

[Pete Moore [:pmoore][:pete]]

Looking into this now...

Comment 21

[Pete Moore [:pmoore][:pete]]

OK, finally have a moment to look at this now (I got caught up in another matter on Tuesday).

Agreed that the repos are mirrored here:
http://git.mozilla.org/?p=external/aosp/platform/external/tcpdump.git;a=summary
http://git.mozilla.org/?p=external/aosp/platform/external/libpcap.git;a=summary

But for some reason they are being looked for here:
http://git.mozilla.org/?p=external/caf/platform/external/tcpdump.git;a=summary
http://git.mozilla.org/?p=external/caf/platform/external/libpcap.git;a=summary

In other words: they are being looked for under "caf" rather than under "aosp". I will now look into the code to see how this path is "built" as the xml entries are:

    <project name="platform/external/libpcap" path="external/libpcap"/>
    <project name="platform/external/tcpdump" path="external/tcpdump"/>

and at the moment I don't know how that gets translated to the full git url path.

Comment 22

[Pete Moore [:pmoore][:pete]]

This is strange indeed, the mappings seem correct:
e.g.: http://hg.mozilla.org/build/mozharness/file/26e71e92141f/configs/b2g_bumper/master.py#l73

This mapping exists in all the b2g2 bumper config files:
http://hg.mozilla.org/build/mozharness/file/26e71e92141f/configs/b2g_bumper

In vcs sync, we do mirror from:
https://android.googlesource.com/platform/external/libpcap/
https://android.googlesource.com/platform/external/tcpdump/

but at the moment I haven't (yet) seen how we know that in b2g_bumper - I will investigate now to see where that is defined...

Comment 23

[Pete Moore [:pmoore][:pete]]

So I think I found it. ;)

base-caf-jb.xml is imported in one and only one place:
 * https://github.com/mozilla-b2g/b2g-manifest/blob/master/flame.xml#L3

In base-caf-jb.xml, no remote="aosp" was specified.

flame.xml specifies a default remote of "caf":
 * https://github.com/mozilla-b2g/b2g-manifest/blob/master/flame.xml#L6

Therefore I'd assume, since no explicit remote is specified, it would take the default, which is "caf" rather than "aosp".

base-jb.xml is imported by four other manifests, which also all have "caf" as the default remote:
 * https://github.com/mozilla-b2g/b2g-manifest/blob/master/nexus-4.xml#L3-L5
 * https://github.com/mozilla-b2g/b2g-manifest/blob/master/emulator-jb.xml#L4-L6
 * https://github.com/mozilla-b2g/b2g-manifest/blob/master/emulator-x86-jb.xml (symlink)
 * https://github.com/mozilla-b2g/b2g-manifest/blob/master/flo.xml#L3-L5

Solution:

It looks to me like you should add remote="aosp" to the new entries for libpcap and tcpdump in base-caf-jb.xml and base-jb.xml.

I would suspect that when you test locally, you will discover you are cloning from:
  * git://codeaurora.org/platform/external/libpcap
  * git://codeaurora.org/platform/external/tcpdump
rather than:
  * https://android.googlesource.com/platform/external/libpcap
  * https://android.googlesource.com/platform/external/tcpdump

I've tested that these codeaurora URLs also work - and they do - which is why you no doubt did not pick this up in testing.

Are you sure you want to mirror from https://android.googlesource.com/ and not git://codeaurora.org/ ?

Comment 24

[Michael Wu [:mwu]]

The mirror on git.mozilla.org should have been from caf, not aosp. CAF is our standard mirror and has all the same things that AOSP has.

Comment 25

[Pete Moore [:pmoore][:pete]]

See https://bugzilla.mozilla.org/show_bug.cgi?id=1029342#c2 - this is not what the request was.

Comment 26

[Pete Moore [:pmoore][:pete]]

If this needs to be changed, probably best to reopen that bug.

Comment 27

[Michael Wu [:mwu]]

Yeah, all the requested mirrors on that bug were wrong. :)

Comment 28

[Pete Moore [:pmoore][:pete]]

OK, I've reopened bug 1029342. Would be grateful if you both (Michael and Alexandre) could check it before we proceed:
https://bugzilla.mozilla.org/show_bug.cgi?id=1029342#c9

Many thanks,
Pete

Comment 29

[:gerard-majax]

Attachment: Manifest update, rebased

New PR, hopefully the last one.

Comment 30

[:gerard-majax]

Merged: https://github.com/mozilla-b2g/b2g-manifest/commit/8c29810e6c896941738de7934ceb50733fbfe35b

Comment 31

[Pete Moore [:pmoore][:pete]]

Working: https://hg.mozilla.org/integration/b2g-inbound/rev/8943899c69e0