Last Comment Bug 512489 - (support-L64) [tracking bug] officially support Linux x86-64 builds
(support-L64)
: [tracking bug] officially support Linux x86-64 builds
Status: RESOLVED FIXED
[linux64]
:
Product: Release Engineering
Classification: Other
Component: Other (show other bugs)
: other
: x86_64 Linux
: P5 normal (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
Depends on: 446330 464750 505232 515457 515612 515616 519072 519074 520684 520722 530680 532560 543844 548605 550086 553365 554021 554028 554138 558595 558739 559057 562422 564164 566026 566269 566275 566279 571470 571578 577154 578749
Blocks: 464027 523365
  Show dependency treegraph
 
Reported: 2009-08-25 10:16 PDT by Ted Mielczarek [:ted.mielczarek]
Modified: 2013-08-12 21:54 PDT (History)
35 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
enable debug builds for Linux 64 (14.68 KB, patch)
2010-02-26 11:57 PST, Armen Zambrano [:armenzg] - Engineering productivity
bhearsum: review-
Details | Diff | Splinter Review
enable debug builds for Linux 64 (14.68 KB, patch)
2010-02-26 12:06 PST, Armen Zambrano [:armenzg] - Engineering productivity
no flags Details | Diff | Splinter Review
enable debug builds for Linux 64 (v3) (14.68 KB, patch)
2010-02-26 12:12 PST, Armen Zambrano [:armenzg] - Engineering productivity
bhearsum: review+
armenzg: checked‑in+
Details | Diff | Splinter Review
disable electrolysis linux64 debug builds (3.54 KB, patch)
2010-02-26 16:24 PST, Armen Zambrano [:armenzg] - Engineering productivity
armenzg: review+
Details | Diff | Splinter Review

Description Ted Mielczarek [:ted.mielczarek] 2009-08-25 10:16:33 PDT
We currently produce Linux x86-64 nightly builds, but not release builds. There's a long thread discussing this on dev.platform:
http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/a44eab1bea078580#

I think it's reasonable to consider officially supporting Linux x86-64 builds, including producing release builds for that platform. Linux distributions are shipping them, and users are using them regardless of what we do, so we might as well acknowledge them. That being said, there are plenty of things that may or may not block this, including:
* x86-64 JIT
* Breakpad support
* Unittests on x86-64
* Talos on x86-64
Comment 1 Benjamin Smedberg AWAY UNTIL 2-AUG-2016 [:bsmedberg] 2009-08-25 11:25:02 PDT
(For the record, discussions about this should take place in mozilla.dev.planning, not here)
Comment 2 Ted Mielczarek [:ted.mielczarek] 2009-08-25 11:31:51 PDT
Yes, this is strictly to track dependencies.
Comment 3 John O'Duinn [:joduinn] (please use "needinfo?" flag) 2009-08-25 12:33:43 PDT
Moving to future while discussions continue. 

Also, as decisions are reached in newsgroups, please start filing dep.bugs for the different parts of all this project.
Comment 4 John O'Duinn [:joduinn] (please use "needinfo?" flag) 2009-12-03 18:25:47 PST
Not sure if bug#523365 is blocking or is something we can do later. Adding here to make sure we dont lose the bug.
Comment 5 John O'Duinn [:joduinn] (please use "needinfo?" flag) 2009-12-03 18:30:33 PST
Currently we only run unittests on a small set of OS - a subset of the OS we support, and also run Talos on.

We have a separate project to run unittests on all OS that we support, but that is not blocking supporting linux64 on par with our other OS.
Comment 6 Ted Mielczarek [:ted.mielczarek] 2009-12-03 20:14:04 PST
John: that's not the same thing. Linux on x86-64 is a different *cpu architecture*, which means we are producing fundamentally different output from the compiler, and using entirely different code in things like Tracemonkey. I don't think anyone in Engineering would be comfortable shipping these builds as officially supported without having our unittest suites run on them.
Comment 7 Benjamin Smedberg AWAY UNTIL 2-AUG-2016 [:bsmedberg] 2009-12-03 20:21:55 PST
I don't think we should do this, FTR, because it looks like releng is taking this bug seriously. Let's focus on things that really matter, like mobile platforms, and skip distractions like this.
Comment 8 Reed Loden [:reed] (use needinfo?) 2009-12-03 20:30:22 PST
(In reply to comment #7)
> I don't think we should do this, FTR, because it looks like releng is taking
> this bug seriously. Let's focus on things that really matter, like mobile
> platforms, and skip distractions like this.

I'm very confused by the above comment. Linux distros are _shipping_ native x86-64 builds of Firefox already, so we should support such builds on our end as well.
Comment 9 Peter Weilbacher 2009-12-04 03:34:40 PST
Indeed, I'm sure that someone has more accurate numbers, but I would think that more people are running FF on Linux (and at least half of those are on x86-64 by now, I posted those numbers in the newsgroups some weeks ago, don't find that message right now) than on mobile. So this is not a "distraction" at all, but important stuff.

I think Benjamin just likes to post provocative statements like this, as if they were an absolute...
Comment 10 Benjamin Smedberg AWAY UNTIL 2-AUG-2016 [:bsmedberg] 2009-12-04 06:23:41 PST
No, I'm absolutely serious. The Mozilla release and QA teams are resource-constrained (people and machines). Just because Linux distros ship an architecture doesn't mean Mozilla itself needs to take responsibility for providing and QAing builds of that architecture.

We could, instead, drop Linux x86 and only do x86-64 builds: that would at least prevent adding another platform to the release and QA cycles.
Comment 11 Ben Hearsum (:bhearsum) 2009-12-04 06:25:52 PST
(In reply to comment #10)
> We could, instead, drop Linux x86 and only do x86-64 builds: that would at
> least prevent adding another platform to the release and QA cycles.

...but still require a ton of upfront work on the build side. *and* it would reduce the number of places we *can* run. 32-bit builds still function on 64-bit platforms, the inverse isn't true.
Comment 12 Peter Weilbacher 2009-12-04 13:24:41 PST
(In reply to comment #10)
> Just because Linux distros ship an
> architecture doesn't mean Mozilla itself needs to take responsibility for
> providing and QAing builds of that architecture.

You got a point there. But from the user perspective this makes little difference. Most distros ship with official branding, even on x86_64, so from what users know, they are using _the_ Mozilla Firefox. After what I have seen in two of the distros, I don't think any of them do unit tests, either, performance tests even less. And they don't document that e.g. JIT isn't active on amd64. They do some QA, but mostly within the constraints (set of libraries) of their distro, assuming other things are QAed by upstream (just like with other packages). So, bad amd64 build quality ultimately reflects on MoCo.
Comment 13 Justin Dolske [:Dolske] 2009-12-04 13:40:18 PST
I'd also point out that until such time as the Linux distros fully take over testing and bustage fixing, Mozilla developers are going to be the ones tasked with fixing x64 regressions. And so from that practical point of view, we should really have full coverage because that job sucks if you have no historical data and don't catch problems early.

There's a few other points I'd make, but I think a newsgroup thread would be more appropriate to continue this discussion -- especially if there's a serious desire to shift the testing burden to distros (which seems like a major policy shift to me). Who wants to start the thread?
Comment 14 Ben Hearsum (:bhearsum) 2010-02-09 12:18:35 PST
Adding breakpad x86-64 bugs as blocking.
Comment 15 Armen Zambrano [:armenzg] - Engineering productivity 2010-02-26 11:57:31 PST
Created attachment 429172 [details] [diff] [review]
enable debug builds for Linux 64

This has been enabled on staging and has been working.
Comment 16 Ben Hearsum (:bhearsum) 2010-02-26 12:00:25 PST
Comment on attachment 429172 [details] [diff] [review]
enable debug builds for Linux 64

You still have ccache in here, please get rid of it.
Comment 17 Armen Zambrano [:armenzg] - Engineering productivity 2010-02-26 12:06:27 PST
Created attachment 429173 [details] [diff] [review]
enable debug builds for Linux 64
Comment 18 Armen Zambrano [:armenzg] - Engineering productivity 2010-02-26 12:12:42 PST
Created attachment 429177 [details] [diff] [review]
enable debug builds for Linux 64 (v3)
Comment 19 Armen Zambrano [:armenzg] - Engineering productivity 2010-02-26 12:31:23 PST
Comment on attachment 429177 [details] [diff] [review]
enable debug builds for Linux 64 (v3)

http://hg.mozilla.org/build/buildbot-configs/rev/9e15aa609448

Masters reconfigured. Triggered some debug builds.
Comment 20 Armen Zambrano [:armenzg] - Engineering productivity 2010-02-26 14:49:21 PST
I am hiding the the debug builds for a couple of branches until I get them fixed:
* electrolysis
* lorentz
Comment 21 Armen Zambrano [:armenzg] - Engineering productivity 2010-02-26 16:24:59 PST
Created attachment 429264 [details] [diff] [review]
disable electrolysis linux64 debug builds

r+ from aki over IRC.

I will have e10n leak builds completely disabled (not just hidden) until the bug 547029 is fixed. If I don't the L64 slaves will stay for long times of periods trying to post to the graph server and therefore causing delays for all other builds.

I will land this later tonight.
Comment 22 Armen Zambrano [:armenzg] - Engineering productivity 2010-02-26 20:14:30 PST
Comment on attachment 429264 [details] [diff] [review]
disable electrolysis linux64 debug builds

There is no need for this patch after all since that builder is not that active (depends on checkins on that branch). We still want the graph patch to land on Monday.
Comment 23 Arpad Borsos [:Swatinem] 2010-02-27 09:30:18 PST
Would someone also please enable scraping on the added box for tbpl goodness? I don't have a Tinderbox password.
Comment 24 Robert Kaiser 2010-02-27 11:06:09 PST
(In reply to comment #23)
> Would someone also please enable scraping on the added box for tbpl goodness?

Done for "Linux x86-64 mozilla-central leak test build" on the "Firefox" tree. Do other trees need that? Which ones?
Comment 25 Armen Zambrano [:armenzg] - Engineering productivity 2010-03-01 09:34:23 PST
(In reply to comment #20)
> I am hiding the the debug builds for a couple of branches until I get them
> fixed:
> * electrolysis
> * lorentz

Electolysis debug builds are now visible and green!
Lorentz will come during this week.
Comment 26 Armen Zambrano [:armenzg] - Engineering productivity 2010-03-03 11:14:45 PST
(In reply to comment #25)
> (In reply to comment #20)
> > I am hiding the the debug builds for a couple of branches until I get them
> > fixed:
> > * electrolysis
> > * lorentz
> 
> Electolysis debug builds are now visible and green!
> Lorentz will come during this week.

Lorentz is now available as well.
Comment 27 Ben Hearsum (:bhearsum) 2010-03-08 06:32:19 PST
bug 546454 no longer blocks this because our upgrade to gcc 4.3.3 works around the issue.
Comment 28 Ben Hearsum (:bhearsum) 2010-03-22 05:41:15 PDT
Armen has been driving this work, re-assigning this bug.
Comment 29 Armen Zambrano [:armenzg] - Engineering productivity 2010-05-10 10:59:05 PDT
Bug 550086 is for adding try linux64 slaves.
Comment 30 Armen Zambrano [:armenzg] - Engineering productivity 2010-05-11 07:50:41 PDT
Status update:
* releases - we will include this for our alpha/beta cycles for FF4 (I have put this on the side)
* unit tests - this is on-going, there is not too much left and we should be completed before EOQ
* try builders - this is on-going, there is not too much left and we should be completed before EOQ
Comment 31 Armen Zambrano [:armenzg] - Engineering productivity 2010-09-01 13:11:58 PDT
tracking bug -> P5
Comment 32 Ben Hearsum (:bhearsum) 2010-11-25 07:54:33 PST
It seems like we already "official support" these builds. There's a couple of tiny issues still, but we've done many releases off of these and have some degree of build and test pool, which is as supported as other official platforms.

IMHO this is fixed.
Comment 33 Xavier Robin 2010-11-25 08:19:05 PST
It isn't fixed as it is still (nearly) impossible to download the 64bits version from the usual places (like getfirefox.com).
Comment 34 Ben Hearsum (:bhearsum) 2010-11-25 08:21:07 PST
That will change when Firefox 4.0 ships.

This bug is tracking officially supported Linux 64-bit builds from a Release Engineering standpoint, which is complete.

Note You need to log in before you can comment on or make changes to this bug.