Bug 512489 (support-L64)

[tracking bug] officially support Linux x86-64 builds

RESOLVED FIXED

Status

Release Engineering
Other
P5
normal
RESOLVED FIXED
8 years ago
4 years ago

People

(Reporter: ted, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [linux64])

Attachments

(1 attachment, 3 obsolete attachments)

(Reporter)

Description

8 years ago
We currently produce Linux x86-64 nightly builds, but not release builds. There's a long thread discussing this on dev.platform:
http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/a44eab1bea078580#

I think it's reasonable to consider officially supporting Linux x86-64 builds, including producing release builds for that platform. Linux distributions are shipping them, and users are using them regardless of what we do, so we might as well acknowledge them. That being said, there are plenty of things that may or may not block this, including:
* x86-64 JIT
* Breakpad support
* Unittests on x86-64
* Talos on x86-64
(For the record, discussions about this should take place in mozilla.dev.planning, not here)
(Reporter)

Comment 2

8 years ago
Yes, this is strictly to track dependencies.
Moving to future while discussions continue. 

Also, as decisions are reached in newsgroups, please start filing dep.bugs for the different parts of all this project.
Component: Release Engineering → Release Engineering: Future
Depends on: 464027
Whiteboard: abc
Whiteboard: abc
Depends on: 515457
Depends on: 515616
Depends on: 515612
Depends on: 520684, 519072, 519074, 520722
Depends on: 532560
Not sure if bug#523365 is blocking or is something we can do later. Adding here to make sure we dont lose the bug.
Depends on: 530680
Currently we only run unittests on a small set of OS - a subset of the OS we support, and also run Talos on.

We have a separate project to run unittests on all OS that we support, but that is not blocking supporting linux64 on par with our other OS.
Blocks: 520722
No longer depends on: 520722
Blocks: 464027
No longer depends on: 464027
(Reporter)

Comment 6

8 years ago
John: that's not the same thing. Linux on x86-64 is a different *cpu architecture*, which means we are producing fundamentally different output from the compiler, and using entirely different code in things like Tracemonkey. I don't think anyone in Engineering would be comfortable shipping these builds as officially supported without having our unittest suites run on them.
I don't think we should do this, FTR, because it looks like releng is taking this bug seriously. Let's focus on things that really matter, like mobile platforms, and skip distractions like this.
(In reply to comment #7)
> I don't think we should do this, FTR, because it looks like releng is taking
> this bug seriously. Let's focus on things that really matter, like mobile
> platforms, and skip distractions like this.

I'm very confused by the above comment. Linux distros are _shipping_ native x86-64 builds of Firefox already, so we should support such builds on our end as well.

Comment 9

8 years ago
Indeed, I'm sure that someone has more accurate numbers, but I would think that more people are running FF on Linux (and at least half of those are on x86-64 by now, I posted those numbers in the newsgroups some weeks ago, don't find that message right now) than on mobile. So this is not a "distraction" at all, but important stuff.

I think Benjamin just likes to post provocative statements like this, as if they were an absolute...
No, I'm absolutely serious. The Mozilla release and QA teams are resource-constrained (people and machines). Just because Linux distros ship an architecture doesn't mean Mozilla itself needs to take responsibility for providing and QAing builds of that architecture.

We could, instead, drop Linux x86 and only do x86-64 builds: that would at least prevent adding another platform to the release and QA cycles.
(In reply to comment #10)
> We could, instead, drop Linux x86 and only do x86-64 builds: that would at
> least prevent adding another platform to the release and QA cycles.

...but still require a ton of upfront work on the build side. *and* it would reduce the number of places we *can* run. 32-bit builds still function on 64-bit platforms, the inverse isn't true.

Comment 12

8 years ago
(In reply to comment #10)
> Just because Linux distros ship an
> architecture doesn't mean Mozilla itself needs to take responsibility for
> providing and QAing builds of that architecture.

You got a point there. But from the user perspective this makes little difference. Most distros ship with official branding, even on x86_64, so from what users know, they are using _the_ Mozilla Firefox. After what I have seen in two of the distros, I don't think any of them do unit tests, either, performance tests even less. And they don't document that e.g. JIT isn't active on amd64. They do some QA, but mostly within the constraints (set of libraries) of their distro, assuming other things are QAed by upstream (just like with other packages). So, bad amd64 build quality ultimately reflects on MoCo.
I'd also point out that until such time as the Linux distros fully take over testing and bustage fixing, Mozilla developers are going to be the ones tasked with fixing x64 regressions. And so from that practical point of view, we should really have full coverage because that job sucks if you have no historical data and don't catch problems early.

There's a few other points I'd make, but I think a newsgroup thread would be more appropriate to continue this discussion -- especially if there's a serious desire to shift the testing burden to distros (which seems like a major policy shift to me). Who wants to start the thread?
Whiteboard: [linux64]
Assignee: nobody → bhearsum
Adding breakpad x86-64 bugs as blocking.
Depends on: 464750, 543844
Blocks: 523365
Assignee: bhearsum → nobody
Component: Release Engineering: Future → Release Engineering
Assignee: nobody → bhearsum
Depends on: 546454
Depends on: 505232
No longer blocks: 520722
Depends on: 520722
Depends on: 548605
Created attachment 429172 [details] [diff] [review]
enable debug builds for Linux 64

This has been enabled on staging and has been working.
Attachment #429172 - Flags: review?(bhearsum)
Comment on attachment 429172 [details] [diff] [review]
enable debug builds for Linux 64

You still have ccache in here, please get rid of it.
Attachment #429172 - Flags: review?(bhearsum) → review-
Created attachment 429173 [details] [diff] [review]
enable debug builds for Linux 64
Attachment #429172 - Attachment is obsolete: true
Attachment #429173 - Flags: review?(bhearsum)
Created attachment 429177 [details] [diff] [review]
enable debug builds for Linux 64 (v3)
Attachment #429173 - Attachment is obsolete: true
Attachment #429177 - Flags: review?(bhearsum)
Attachment #429173 - Flags: review?(bhearsum)
Attachment #429177 - Flags: review?(bhearsum) → review+
Comment on attachment 429177 [details] [diff] [review]
enable debug builds for Linux 64 (v3)

http://hg.mozilla.org/build/buildbot-configs/rev/9e15aa609448

Masters reconfigured. Triggered some debug builds.
Attachment #429177 - Flags: checked-in+
I am hiding the the debug builds for a couple of branches until I get them fixed:
* electrolysis
* lorentz
Created attachment 429264 [details] [diff] [review]
disable electrolysis linux64 debug builds

r+ from aki over IRC.

I will have e10n leak builds completely disabled (not just hidden) until the bug 547029 is fixed. If I don't the L64 slaves will stay for long times of periods trying to post to the graph server and therefore causing delays for all other builds.

I will land this later tonight.
Attachment #429264 - Flags: review+
Comment on attachment 429264 [details] [diff] [review]
disable electrolysis linux64 debug builds

There is no need for this patch after all since that builder is not that active (depends on checkins on that branch). We still want the graph patch to land on Monday.
Would someone also please enable scraping on the added box for tbpl goodness? I don't have a Tinderbox password.

Comment 24

7 years ago
(In reply to comment #23)
> Would someone also please enable scraping on the added box for tbpl goodness?

Done for "Linux x86-64 mozilla-central leak test build" on the "Firefox" tree. Do other trees need that? Which ones?
(In reply to comment #20)
> I am hiding the the debug builds for a couple of branches until I get them
> fixed:
> * electrolysis
> * lorentz

Electolysis debug builds are now visible and green!
Lorentz will come during this week.
(In reply to comment #25)
> (In reply to comment #20)
> > I am hiding the the debug builds for a couple of branches until I get them
> > fixed:
> > * electrolysis
> > * lorentz
> 
> Electolysis debug builds are now visible and green!
> Lorentz will come during this week.

Lorentz is now available as well.
bug 546454 no longer blocks this because our upgrade to gcc 4.3.3 works around the issue.
No longer depends on: 546454
Alias: support-L64
Summary: officially support Linux x86-64 builds → [tracking bug] officially support Linux x86-64 builds
Attachment #429264 - Attachment is obsolete: true
(Reporter)

Updated

7 years ago
Depends on: 553365
Armen has been driving this work, re-assigning this bug.
Assignee: bhearsum → armenzg
(Reporter)

Updated

7 years ago
Depends on: 554021
(Reporter)

Updated

7 years ago
Depends on: 554028
(Reporter)

Updated

7 years ago
Depends on: 554138
Priority: -- → P2
(Reporter)

Updated

7 years ago
Depends on: 555674
(Reporter)

Updated

7 years ago
No longer depends on: 555674

Updated

7 years ago
Depends on: 558739
Depends on: 559057

Updated

7 years ago
Depends on: 446330

Updated

7 years ago
Depends on: 558595
Depends on: 564164
Bug 550086 is for adding try linux64 slaves.
Depends on: 550086
Status update:
* releases - we will include this for our alpha/beta cycles for FF4 (I have put this on the side)
* unit tests - this is on-going, there is not too much left and we should be completed before EOQ
* try builders - this is on-going, there is not too much left and we should be completed before EOQ
Priority: P2 → P4
Depends on: 566279
Depends on: 566275
Depends on: 566026
Depends on: 566269
Depends on: 571143
No longer depends on: 571143
Depends on: 571331

Updated

7 years ago
Depends on: 571578
Depends on: 571470
Depends on: 562422
Depends on: 577154
Depends on: 578749
No longer depends on: 571331
tracking bug -> P5
Assignee: armenzg → nobody
Priority: P4 → P5
It seems like we already "official support" these builds. There's a couple of tiny issues still, but we've done many releases off of these and have some degree of build and test pool, which is as supported as other official platforms.

IMHO this is fixed.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED

Comment 33

7 years ago
It isn't fixed as it is still (nearly) impossible to download the 64bits version from the usual places (like getfirefox.com).
That will change when Firefox 4.0 ships.

This bug is tracking officially supported Linux 64-bit builds from a Release Engineering standpoint, which is complete.
(Assignee)

Updated

4 years ago
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.