1456164 - Snappy builds aren't on buildhub

Status: RESOLVED WORKSFORME

(Release Engineering :: Release Automation: Snap, defect)

Description

[William Lachance (:wlach)]

Hey, it seems like there has been some headway on building snappy artifacts and uploading them to the ubuntu store in bug 1297513. However, I can't seem to figure out where they are actually generated on the mozilla-release branch on treeherder, nor do I see any evidence of them on buildhub (https://mozilla-services.github.io/buildhub/)

From the perspective of mission control and release health monitoring via telemetry, it would be nice to be able to identify "official" Mozilla Linux snappy builds once they represent a non-trivial quantity of users. Currently we have a big mismash of submissions from builds that are not under our control, which is difficult to understand and visualize.

Do we have plans for this? This is also relevant for flatpak, bug 1278719. Is bug 1443873 relevant to this?

Comment 1

[Johan Lorenzo [:jlorenzo]]

Thanks for the call out! I don't remember doing anything related to buildhub, so that's something we should do.

Snaps are currently built on beta and release only. ESR is upcoming in 60, and nightly will be eventually be done. Results are reported on Treeherder[1]. I realize the group name isn't display. I'll fix it up. 

Regarding Telemetry, I know Snap build are tagged as a partner repack[2]. Is that enough to identify a Snap? For the record, the Snap store also give us data about how many users we have per version. However, it's good to have all usage data under one system. 

What would we need to get the builds under buildhub and Telemetry? 

[1] https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=f2ac3383fb97a55d11a876f17189613a852a0077&filter-searchStr=snap&selectedJob=175202164
[2] https://github.com/mozilla-partners/canonical/blob/96b444c1d1961b1396d38f3c651ba910c6d97859/desktop/ubuntu/distribution/distribution.ini#L6

Comment 2

[William Lachance (:wlach)]

(In reply to Johan Lorenzo [:jlorenzo] from comment #1)
> Thanks for the call out! I don't remember doing anything related to
> buildhub, so that's something we should do.
> 
> Regarding Telemetry, I know Snap build are tagged as a partner repack[2]. Is
> that enough to identify a Snap? For the record, the Snap store also give us
> data about how many users we have per version. However, it's good to have
> all usage data under one system. 

Should be, it looks like partner id is part of the telemetry environment. https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html?highlight=partnerid

> What would we need to get the builds under buildhub and Telemetry? 

I assume we would have to add support for getting the information to buildhub, it looks like this is the code that currently scrapes that:

https://github.com/mozilla-services/buildhub/blob/master/jobs/buildhub/inventory_to_records.py

It looks like that code is scraping records from http://archive.mozilla.org/ Do we upload the snaps there? I'm not seeing anything snappy related checking e.g. http://archive.mozilla.org/pub/firefox/releases/59.0.2/

I suspect from a buildhub perspective it would be easiest if you did something like that, not sure how difficult it would be.

Comment 3

[Mike Kaply [:mkaply]]

Yes, these builds have a distribution ID, so it should be easy to see in telemetry.

I've also recently bumped the distribution ID for the Snap, so we should be able to distinguish the Snap from other canonical builds.

Comment 4

[Johan Lorenzo [:jlorenzo]]

(In reply to William Lachance (:wlach) (use needinfo!) from comment #2)
> It looks like that code is scraping records from http://archive.mozilla.org/
> Do we upload the snaps there? I'm not seeing anything snappy related
> checking e.g. http://archive.mozilla.org/pub/firefox/releases/59.0.2/

I see what's wrong. We used to upload snaps on archive.mozilla.org, but on the candidates directory[1]. That's something I missed when we moved the task definition in tree. I'll fix it next week (I'm currently in a work week unrelated to snaps). 

[1] Eg: https://archive.mozilla.org/pub/firefox/candidates/56.0b2-candidates/build1/snap/

Comment 5

[William Lachance (:wlach)]

Johan: Ping. Any chance this could be fixed? It would be good to be able to start tracking crash rates of snaps inside mission control (which would be implicitly enabeld by adding the snaps to buildhub):

https://data-missioncontrol.dev.mozaws.net/#/release/linux/main_crashes

Comment 6

[Johan Lorenzo [:jlorenzo]]

I'm sorry, I think releng doesn't have the bandwidth to get something out this quarter. I'll keep you tuned.

Comment 7

[William Lachance (:wlach)]

:catlee pointed out that the snaps have the same buildid as standard linux release builds, so the data in buildhub basically already accounts for them. I verified that we're seeing distinct telemetry for snappy builds that have the expected build ids associated with them:

https://sql.telemetry.mozilla.org/queries/54789/source#144484

Comment 8

[Johan Lorenzo [:jlorenzo]]

Excellent! :D