1152871 - Add releng build file containing Adjust SDK private app token

Status: RESOLVED FIXED

(Firefox Build System :: Android Studio and Gradle Integration, defect, P2)

Description

[Nick Alexander :nalexander [he/him]]

In order to use the Adjust SDK to track Fennec installations, we need to identify ourselves to the Adjust endpoint with a private app token.  We should follow the precedent around [1] and add a configure flag for specifying a file containing the token.  (And then make sure we don't leak the variable to the console log during the build.)  The patches in Bug 1143888 do this work [2].

This ticket tracks:

* acquiring an Adjust SDK app token;
* conveying it to releng;
* making it available to the automation builders (in /build or wherever is good for Fennec builds).

This process is exactly the same as was done in [3], but I can't determine who actually did the releng work!

[1] https://dxr.mozilla.org/mozilla-central/source/configure.in#4071

[2] We could land the new build flags, etc, separately from the actual build and SDK integration, if we want to.

[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1069816#c6

Comment 1

[Nick Alexander :nalexander [he/him]]

mfinkle: you're on the hook to acquire the Adjust SDK app token.

coop: could you redirect to the releng point person for arranging for a new file to be available to mobile/android builds?

Comment 2

[Chris Cooper [:coop] (he/him)]

(In reply to Nick Alexander :nalexander from comment #1)
> coop: could you redirect to the releng point person for arranging for a new
> file to be available to mobile/android builds?

I've done this recently enough myself that I can be the one on the hook for this.

Comment 3

[Mark Finkle (:mfinkle) (use needinfo?)]

I created an apptoken for Firefox on the Adjust system. How should I convey the token to it's private location?

Comment 4

[Chris Cooper [:coop] (he/him)]

(In reply to Mark Finkle (:mfinkle) from comment #3)
> I created an apptoken for Firefox on the Adjust system. How should I convey
> the token to it's private location?

Mark: you can encrypt it and send it to me. I'll add it to the repo.

http://gpg.mozilla.org/pks/lookup?search=coop%40mozilla.com

Comment 5

[Chris Cooper [:coop] (he/him)]

I've added the token to the hiera cache. 

Next step is a series of patches to buildbot-configs, mozharness, and puppet to get the token installed on the mobile builders.

Comment 6

[Nick Alexander :nalexander [he/him]]

(In reply to Chris Cooper [:coop] from comment #5)
> I've added the token to the hiera cache. 
> 
> Next step is a series of patches to buildbot-configs, mozharness, and puppet
> to get the token installed on the mobile builders.

Can we get an update here?  I'm not familiar with the mobile beta schedule but I do think we're very short for time.  This will also require a change to the beta and release mozconfig's (in tree?) to specify --with-adjust-sdk-keyfile [1].

[1] https://dxr.mozilla.org/mozilla-central/source/configure.in#4033

Comment 7

[Chris Cooper [:coop] (he/him)]

Sorry, I didn't realize this was a fire drill. The blocked bug is resolved, and there's no other indicator of timing.

Planning on getting the patches written today.

Comment 8

[Nick Alexander :nalexander [he/him]]

(In reply to Chris Cooper [:coop] from comment #7)
> Sorry, I didn't realize this was a fire drill. The blocked bug is resolved,
> and there's no other indicator of timing.

Hey coop, sorry, that's on me.

> Planning on getting the patches written today.

I'm happy to do the in-tree stuff.

Comment 9

[Chris Cooper [:coop] (he/him)]

Attachment: [puppet] Add Adjust SDK token for CentOS

Pinned an ec2 slave to my puppet env and successfully ended up with the Adjust token under /builds.

Comment 10

[Chris Cooper [:coop] (he/him)]

Attachment: [buildbot-configs] Copy Adjust token into the mock env for Android builds

Comment 11

[Chris Cooper [:coop] (he/him)]

Attachment: [mozharness] Copy Adjust token into the mock env

Comment 12

[Chris Cooper [:coop] (he/him)]

(In reply to Nick Alexander :nalexander from comment #8)
> I'm happy to do the in-tree stuff.

The token will live alongside the others in /builds: /builds/adjust-sdk.token

Comment 13

[Justin Wood (:Callek)]

Comment on attachment 8598291 [details] [diff] [review]
[buildbot-configs] Copy Adjust token into the mock env for Android builds

Review of attachment 8598291 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla/config.py
@@ +1489,5 @@
>                  ('/home/cltbld/.boto', '/builds/.boto'),
>                  ('/builds/mozilla-api.key', '/builds/mozilla-api.key'),
>                  ('/builds/mozilla-fennec-geoloc-api.key', '/builds/mozilla-fennec-geoloc-api.key'),
>                  ('/builds/crash-stats-api.token', '/builds/crash-stats-api.token'),
> +                ('/builds/adjust-sdk.token', '/builds/adjust-sdk.token'),

we're getting pretty thick here, might be worth us considering a sub-folder for this stuff....

Comment 14

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8598287 [details] [diff] [review]
[puppet] Add Adjust SDK token for CentOS

Review of attachment 8598287 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/puppet/rev/d1c288081126

Comment 15

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8598291 [details] [diff] [review]
[buildbot-configs] Copy Adjust token into the mock env for Android builds

Review of attachment 8598291 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/buildbot-configs/rev/2d9e77a87dfa

Comment 16

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8598294 [details] [diff] [review]
[mozharness] Copy Adjust token into the mock env

Review of attachment 8598294 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/mozharness/rev/c9c61c6fee9b

Comment 17

[Chris Cooper [:coop] (he/him)]

(In reply to Chris Cooper [:coop] from comment #16) 
> https://hg.mozilla.org/build/mozharness/rev/c9c61c6fee9b

To help avoid a possible back-out, I've kicked off golden AMI generation for the linux build and try platforms to make sure the new token is available on the base system.

Comment 18

[Chris Cooper [:coop] (he/him)]

(In reply to Chris Cooper [:coop] from comment #17) 
> To help avoid a possible back-out, I've kicked off golden AMI generation for
> the linux build and try platforms to make sure the new token is available on
> the base system.

AMIs are ready, so instances should start picking them up.

Comment 19

[Andrew Halberstadt [:ahal]]

Fyi, I believe this caused the failure from bug 1159414. From comment 17 it looks like this depends on some external piece, so maybe it just wasn't ready yet when Chris and I pushed, so good chance there's no problem. Just wanted to point it out in case.

Comment 20

[Nick Alexander :nalexander [he/him]]

In automation [1], I see helpful lines like:

mock_mozilla -r mozilla-centos6-x86_64-android --cwd / --shell '/usr/bin/env  chown -R mock_mozilla /builds/adjust-sdk.token'

[1] http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/fx-team-android-api-11-debug/1430316531/fx-team-android-api-11-debug-bm77-build1-build188.txt.gz

I'm going to prepare the in-tree patch to use the SDK token and push to try.

Comment 21

[Nick Alexander :nalexander [he/him]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=79a205a053b6

Comment 22

[Nick Alexander :nalexander [he/him]]

Attachment: MozReview Request: bz://1152871/nalexander

/r/7871 - Bug 1152871 - Include Adjust SDK private token in builds. r=mfinkle

Pull down this commit:

hg pull -r 79a205a053b664f980836e9c63576746266f79ef https://reviewboard-hg.mozilla.org/gecko/

Comment 23

[Nick Alexander :nalexander [he/him]]

mfinkle: review is tiny, but I realize now we should change the Adjust SDK log level depending on production/sandbox.  Will update.

To test this, I will manually verify (using the Java debugger) that the Adjust SDK actually gets instantiated and uploads a packet.  I'll verify the packet contents for the documentation, just like rnewman forwarded to me.

Comment 24

[Mark Finkle (:mfinkle) (use needinfo?)]

Comment on attachment 8599404 [details]
MozReview Request: bz://1152871/nalexander

https://reviewboard.mozilla.org/r/7869/#review6645

Ship It!

Comment 25

[Nick Alexander :nalexander [he/him]]

https://hg.mozilla.org/integration/fx-team/rev/4767a0a19a62

Comment 26

[Nick Alexander :nalexander [he/him]]

Comment on attachment 8599404 [details]
MozReview Request: bz://1152871/nalexander

Approval Request Comment
[Feature/regressing bug #]: Adjust install tracking.
[User impact if declined]: marketing will be unhappy.
[Describe test coverage new/current, TreeHerder]: none.
[Risks and why]: very low.
[String/UUID change made/needed]: none.

This is required on 38.0.5  This patch flips the switch on the code landed in Bug 1143888.  That code is already uplifted to 38.0.5.

Comment 27

[Nick Alexander :nalexander [he/him]]

Comment on attachment 8599404 [details]
MozReview Request: bz://1152871/nalexander

(See earlier message.)

Comment 28

[Nick Alexander :nalexander [he/him]]

I've pushed a Nightly-like try build with MOZ_INSTALL_TRACKING turned on to

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6088d0c2c3b4

To test, you should be able to just install the APK and start Nightly.  You'll see very minimal logging with the log tag "Adjust" (minimal since there's tons of PII in the Adjust verbose logging).

With a locally built version, I see a single install when viewing the dashboard at adjust.com! \o/

Comment 29

[Nick Alexander :nalexander [he/him]]

Even better, I can see an install into the sandbox environment, which corresponds to a custom local build that I did.  wbowden, over to you to install (and start) the try build and figure out if your installs get to adjust.com as well.

Comment 30

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/4767a0a19a62

Comment 31

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/03c37cb5965b

Comment 32

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/ac5b76223f2f

Comment 34

[Nick Alexander :nalexander [he/him]]

Attachment: MozReview Request: Bug 1152871 - Include Adjust SDK private token in builds. r=mfinkle