866754 - Creation of regular ARMv7 builds with yandex/tstore distribution IDs

Status: RESOLVED DUPLICATE of bug 1286944

(Release Engineering :: Release Requests, defect, P3)

Description

[Alex Keybl [:akeybl]]

For sake of metrics, we'd like to create an ARMv7 APK repack/build with a yandex distribution ID. This will allow us to gather metrics on yandex users separately from Google Play users.

mfinkle brought up the concern of users being automatically updated by Google Play when installed through Yandex. This is something we should test for, to make sure we understand the behavior for future metrics.

Verification of this bug will require ensuring that updates are unaffected by the distribution ID (yandex->yandex, yandex->google, google->yandex), that the distribution ID is maintained when updating yandex->yandex, and that the distribution ID is showing up in metrics.

This bug is not urgent (it does not block release of APKs to the Yandex.Store), but is desirable.

Comment 1

[Anurag Phadke[:[e-mail]]]

:alex - how exactly would we be able to know the previous and current distribution id to understand the transition? AFAIK, the blocklist ping only gives us the "current" distribution id.

-anurag

Comment 2

[[e-mail] (:bhearsum)]

Mark, IIRC you were saying that we can drop a file somewhere in the package to do this? Which file is that?

Comment 3

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to Ben Hearsum [:bhearsum] from comment #2)
> Mark, IIRC you were saying that we can drop a file somewhere in the package
> to do this? Which file is that?

bhearsum, did you see the drop-a-file for custom-channel-for-mobile work in bug#834194?

Comment 4

[[e-mail] (:bhearsum)]

(In reply to John O'Duinn [:joduinn] from comment #3)
> (In reply to Ben Hearsum [:bhearsum] from comment #2)
> > Mark, IIRC you were saying that we can drop a file somewhere in the package
> > to do this? Which file is that?
> 
> bhearsum, did you see the drop-a-file for custom-channel-for-mobile work in
> bug#834194?

Yep, but that's a file in the system, not a file in the package AFAIK.

Comment 5

[Mark Finkle (:mfinkle) (use needinfo?)]

Ben - Here is some info on the distribution system. We allow "in APK" and "in system folder" approaches:
https://wiki.mozilla.org/Mobile/Distribution_Files

Comment 6

[[e-mail] (:bhearsum)]

Thanks Mark!

I'm away most of this week, if someone picks this up before me, this should get you a signed build after you repack it:
hg clone http://hg.mozilla.org/build/tools
cd tools/release/signing
python get_token -c host.cert -u cltsign -p [token generation password] signing1.build.scl1.mozilla.com -d 600 -i [ip of buildbot master] > token
python signtool.py -t token -n nonce -c host.cert -H signing1.build.scl1.mozilla.com -f jar fennec.apk

Comment 7

[Alex Keybl [:akeybl]]

(In reply to aphadke from comment #1)
> :alex - how exactly would we be able to know the previous and current
> distribution id to understand the transition? AFAIK, the blocklist ping only
> gives us the "current" distribution id.

It'd be lost info for now. We can file a blocked bug to make sure the initial distribution ID is somehow maintained for future use.

Comment 8

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Alex Keybl [:akeybl] from comment #7)
> (In reply to aphadke from comment #1)
> > :alex - how exactly would we be able to know the previous and current
> > distribution id to understand the transition? AFAIK, the blocklist ping only
> > gives us the "current" distribution id.
> 
> It'd be lost info for now. We can file a blocked bug to make sure the
> initial distribution ID is somehow maintained for future use.

I was spreading mis-information on this point. Here are some cases:
1. Installed from Yandex -> Has yandex distribution id
2. Installed from Play Store -> Does not have yandex distribution id
3. Installed from Yandex, but upgraded from Play Store -> Keeps yandex distribution id
4. Installed from Yandex, user uninstalls and installs from Play Store -> Does not keep yandex distribution id
5. Installed from Yandex, user clicks "Clear Data" -> Does not keep yandex distribution id

I hope this helps clear the different use cases up.

Comment 9

[Aaron Train [:aaronmt]]

Is this tracking for Fx21?

Comment 10

[[e-mail] (:bhearsum)]

OK, I've got a repack with "yandex" set as the distribution id. It's available for testing from https://people.mozilla.com/~bhearsum/fennec-20.0.1.multi.yandex.apk. Who's on the hook for that?

Comment 11

[Mark Finkle (:mfinkle) (use needinfo?)]

Mobile QA could take a look at about:config and see what distribution.id and version are being reported. Remember, those values need to appear as "default" so they should not be 'bold' or reset-able.

Margaret might want to take a peek too.

Comment 12

[Aaron Train [:aaronmt]]

Off the bat, using the build above I don't see the preference in about:config.

Comment 13

[[e-mail] (:bhearsum)]

Here's a partial look at the unpacked package, for easy viewing:
(venv)➜  fennec  find . | sort
.
./AndroidManifest.xml
./application.ini
./chrome.manifest
./classes.dex
./components
./components/binary.manifest
./distribution
./distribution/preferences.json
./lib
./lib/armeabi-v7a
./lib/armeabi-v7a/libmozglue.so
./lib/armeabi-v7a/libomxpluginfroyo.so
./lib/armeabi-v7a/libomxpluginhc.so
./lib/armeabi-v7a/libomxpluginjb-htc.so
./lib/armeabi-v7a/libomxplugin.so
./lib/armeabi-v7a/libomxpluginsony.so
./lib/armeabi-v7a/libomxplugingb235.so
./lib/armeabi-v7a/libomxplugingb.so
./lib/armeabi-v7a/libplugin-container.so
./libfreebl3.so
./lib.id
./libmozalloc.so
./libmozsqlite3.so
./libnspr4.so
./libnss3.so
./libnssckbi.so
./libnssutil3.so
./libplc4.so
./libplds4.so
./libsmime3.so
./libsoftokn3.so
./libssl3.so
./libxpcom.so
./libxul.so
./META-INF
./META-INF/MANIFEST.MF
./META-INF/RELEASE.RSA
./META-INF/RELEASE.SF
./modules
./modules/accessibility
./modules/accessibility/.mkdir.done
./modules/commonjs
./modules/commonjs/.mkdir.done
./modules/commonjs/promise
./modules/commonjs/promise/.mkdir.done
./modules/identity
./modules/identity/.mkdir.done
./modules/services
./modules/services-common
./modules/services-common/.mkdir.done
./modules/services-crypto
./modules/services-crypto/.mkdir.done
./modules/services/datareporting
./modules/services/datareporting/.mkdir.done
./omni.ja
./package-name.txt
./platform.ini
./recommended-addons.json
./removed-files
./res
...
(venv)➜  fennec  cat application.ini 
; This file is not used. If you modify it and want the application to use
; your modifications, start with the "-app /path/to/application.ini"
; argument.
[App]
Vendor=Mozilla
Name=Fennec
Version=20.0.1
BuildID=20130410154516
SourceRepository=http://hg.mozilla.org/releases/mozilla-release
SourceStamp=48255272a995
ID={aa3c5121-dab2-40e2-81ca-7ea25febc110}

[Gecko]
MinVersion=20.0.1
MaxVersion=20.0.1

[XRE]
EnableExtensionManager=1

[Crash Reporter]
Enabled=1
ServerURL=https://crash-reports.mozilla.com/submit?id={aa3c5121-dab2-40e2-81ca-7ea25febc110}&version=20.0.1&buildid=20130410154516
(venv)➜  fennec  cat distribution/preferences.json 
{
    "Global": {
        "id": "yandextest",
        "version": "1.0"
    }
}

I just realized that this code may not be in 20.0.1 though...

Comment 14

[[e-mail] (:bhearsum)]

What are the next steps here?

Comment 15

[Alex Keybl [:akeybl]]

Margaret - can you provide guidance here?

We'll leave this on the FF22/23 tracking nom lists, since it'd be nice to get this into one of these two upcoming releases.

Comment 16

[:Margaret Leibovic]

Testing the build in comment 12, I also don't see the pref in about:config. The distribution code was uplifted to 19, so that shouldn't be the problem (see bug 834681).

In the error console I see:
E/GeckoConsole( 4804): [JavaScript Error: "Distribution: missing or incomplete Global preferences" {file: "chrome://browser/content/browser.js" line: 8130}]

Looking into the code, you need to supply "id"/"version"/"about" properties:
http://mxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/browser.js#7244

So to fix this, you should add an "about" to your "Global" preferences.

Comment 17

[Mark Finkle (:mfinkle) (use needinfo?)]

Margaret - Can you add tat info to https://wiki.mozilla.org/Mobile/Distribution_Files ?

Comment 18

[:Margaret Leibovic]

(In reply to Mark Finkle (:mfinkle) from comment #17)
> Margaret - Can you add tat info to
> https://wiki.mozilla.org/Mobile/Distribution_Files ?

It is there:
https://wiki.mozilla.org/Mobile/Distribution_Files#Preferences

I guess we could make it more obvious that the properties that aren't marked as (optional) are actually required.

Comment 19

[[e-mail] (:bhearsum)]

I did a new repack with "about" set. It's available here: https://people.mozilla.com/~bhearsum/fennec-20.0.1.multi.yandex-2.apk. I think you're QA'ing this, Aaron?

Comment 20

[Aaron Train [:aaronmt]]

Looks good to me,

distribution.about; Yandex distribution test
distribution.id; yandextest

and

[distribution/preferences.json]
{
    "Global": {
        "id": "yandextest",
        "version": "1.0",
        "about": "Yandex distribution test"
    }
}

Comment 21

[[e-mail] (:bhearsum)]

Alex, is there anything left to do here? Do we know whether we'll be shipping this in 23?

Comment 22

[Alex Keybl [:akeybl]]

(In reply to Ben Hearsum [:bhearsum] from comment #21)
> Alex, is there anything left to do here? Do we know whether we'll be
> shipping this in 23?

We'd like to ship this as part of FF23 but it's not a requirement - this bug was meant to track ongoing creation of this repack. Let's leave it open for now.

Comment 23

[[e-mail] (:bhearsum)]

I'm unassigning myself from this because there's nothing actionable right now, and I don't want to have people ignore this bug if I'm away the next time something needs to happen.

Comment 24

[bhavana bajaj [:bajaj]]

Given https://bugzilla.mozilla.org/show_bug.cgi?id=879328#c8 , I am setting the tracking nom for Firefox 25, 26

Comment 25

[Alex Keybl [:akeybl]]

Sounds like we're going to try to get this done for Q4. For now, we're using in-store metrics.

Comment 26

[Rail Aliiev [:rail]]

Mike, I vaguely remember that we don't want to repack/rebuild our released binaries. If this is the case, do we still need this bug?

Comment 27

[Mike Kaply [:mkaply]]

(In reply to Rail Aliiev [:rail] ⌚️ET from comment #26)
> Mike, I vaguely remember that we don't want to repack/rebuild our released
> binaries. If this is the case, do we still need this bug?

We actually do want to have the ability to do distribution repacks, but that's covered in:

https://bugzilla.mozilla.org/show_bug.cgi?id=1286944

It would be generic and we would create a Yandex distribution for their store.

So I would dupe this to the other bug. We would just create a Yandex partner distribution.