Open Bug 1234029 Opened 4 years ago Updated 4 years ago

use debug builds from archives.m.o

Categories

(Testing :: mozregression, defect)

defect
Not set

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: parkouss, Assigned: parkouss)

References

(Blocks 2 open bugs)

Details

Attachments

(1 file)

From bug 1233909 comment 2, it appears that we can find debug builds on archives.m.o. Currently we only find those builds using TaskCluster - and this is limited in time to one year (even less currently). So let's find those using archives.m.o also.
Blocks: 1233909
For firefox and jsshell, provides access to debug builds using
archives.m.o.

Those builds are stored using 'debug' in their filename, and
using TaskCluster is not required anymore.

Example:

mozregression -g 2013-05-05 -b 2013-05-18 -B debug --app jsshell
 0:02.16 LOG: MainThread Bisector INFO Pushlog:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=c8e47b184aba&tochange=6e2789a70f6b

 0:02.16 LOG: MainThread download INFO Downloading build from: https://archive.mozilla.org/pub/firefox/nightly/2013/05/2013-05-12-mozilla-central-debug/jsshell-linux-x86_64.zip
Assignee: nobody → j.parkouss
Status: NEW → ASSIGNED
Attachment #8700390 - Flags: review?(wlachance)
Oh, and looks like we can also find asan builds on archives.m.o, (this time looking at the filename):

https://archive.mozilla.org/pub/firefox/nightly/2014/05/2014-05-12-mozilla-central-debug/
https://archive.mozilla.org/pub/firefox/nightly/2014/05/2014-05-12-03-02-02-mozilla-central/

Looks like it is for debug and opt builds on linux 64 only. Also it seems that it appears somewhere between 2013 and 2014, since I don't see them for 2013-05-18.

Gary, would it be useful for you to add support for this ?
Flags: needinfo?(gary)
Yes, this will definitely be useful. Thanks for adding this!

As for the earliest known dates with nightlies, I have listed them in bug 1233901, but you might want to double check how best to fix this.
Flags: needinfo?(gary)
Oh - this needs some more thinking maybe. Looks like debug builds are no more on archives.m.o since 2015-11-21:

https://archive.mozilla.org/pub/firefox/nightly/2015/10/

Not sure if that is intentional, or a bug ?

If that is intentional, we should always prefer the TaskCluster method over the server when available.
:coop, do we intend for debug builds to still be on archive.mozilla.org?
Flags: needinfo?(coop)
I would not rely on archive.mozilla.org going forward. The TaskCluster index service will be vastly superior to the monolithic archive/ftp layout.
Flags: needinfo?(coop)
Comment on attachment 8700390 [details] [review]
use debug builds from archives.m.o

It sounds like we might not want to do this, but I'll give this an r+ just in case we do.
Attachment #8700390 - Flags: review?(wlachance) → review+
:coop, Do you know what is the future for keeping builds in a long term approach ? I think taskcluster keep builds one year. What else beyond that ?
Hm, I'm not sure anymore we should land this patch indeed. I mean I can write a workaround - use archive only as a last resort - but still in 6/12 months it will be broken if we don't upload anymore to archive the debug builds. :(

Gary, thoughts ?
Flags: needinfo?(gary)
(In reply to Julien Pagès (:parkouss) from comment #8)
> :coop, Do you know what is the future for keeping builds in a long term
> approach ? I think taskcluster keep builds one year. What else beyond that ?

Another way of querying about this:

(1) We are losing *-debug builds in archive.m.o from 2012/2013 till Nov 2015 if archive.m.o goes away, as they are not found in TaskCluster.
(2) Can TaskCluster please support these builds going forward?
(3) Any plans to still keep those builds in (1) available for access, e.g. via S3?
Flags: needinfo?(gary) → needinfo?(coop)
(In reply to Gary Kwong [:gkw] [:nth10sd] from comment #10)
> (In reply to Julien Pagès (:parkouss) from comment #8)
> > :coop, Do you know what is the future for keeping builds in a long term
> > approach ? I think taskcluster keep builds one year. What else beyond that ?
> 
> Another way of querying about this:
> 
> (1) We are losing *-debug builds in archive.m.o from 2012/2013 till Nov 2015
> if archive.m.o goes away, as they are not found in TaskCluster.
> (2) Can TaskCluster please support these builds going forward?
> (3) Any plans to still keep those builds in (1) available for access, e.g.
> via S3?

My apologies if I'm misstating the plan while :nthomas is on PTO, but we're not planning on losing any existing builds on archive.m.o, it's just that any new debug builds will only be available via TC. The netapp storage that backs archive.m.o is out of warranty.

We may move the existing debug builds on archive.m.o to S3 at some point if we feel ambitious.
Flags: needinfo?(coop)
Thanks for the clarification!

> We may move the existing debug builds on archive.m.o to S3 at some point if
> we feel ambitious.

Yes, please! As long as we have a way of accessing these builds (no matter archive.m.o or S3) that's fine.
So in summary:

2012/2013 till Nov 2015 builds, we should use archive.m.o. (which may move to S3 at some point)
Anything later, we should use TaskCluster.

(In reply to Julien Pagès (:parkouss) from comment #4)
> If that is intentional, we should always prefer the TaskCluster method over
> the server when available.

I would say I agree with you here, yep.
You need to log in before you can comment on or make changes to this bug.