Status

enhancement
RESOLVED FIXED
12 years ago
10 years ago

People

(Reporter: Waldo, Assigned: sayrer)

Tracking

Details

()

Attachments

(2 attachments, 5 obsolete attachments)

A one-stop shop for downloading the latest nightly, with small links to FTP to find previous nightlies, would be awesome for making downloading nightlies easy.  See the URL for an example of how this would work.
Nick, think you could make something like this from your hourly-archive stuff? I could set up nightly.localgho.st for now to let you try it...
Actually, this is closer to update-watch. That already scrapes ftp for nightly builds. 
Duplicate of this bug: 416847
Duplicate of this bug: 356421
Posted file nightly web page contents (obsolete) —
please please please don't bikeshed this to death. we should get something up and running. it's always possible file followups to change it. Most people are looking for Firefox, so that's all this does.

morgamic asked whether it should it be .com or .org... I said "both!"
(In reply to comment #5)
> Created an attachment (id=408443) [details]
> nightly web page contents
> 
> please please please don't bikeshed this to death. 

To clarify, it obviously needs graphic design. But I claim the page contents are good enough to start with. Examples of things I don't think we should address right now are: N.N.N branch builds, Firefox ARM builds, tracemonkey builds, thunderbird builds, fennec builds, shark builds, "what's a nightly?", mozilla.com navbars, the fact that the binary is called Minefield, etc, etc.

All of those things might be good to fix later. Most could be added to a page that the "more stuff" link goes to, as a separate enhancement.

I think this page should be as simple as possible to start, and any changes we make should be done with download metrics in hand.
+Infinity, agree that pretty is relatively easy once it's up and running
(In reply to comment #7)
> +Infinity, agree that pretty is relatively easy once it's up and running

Oh, we can even make it pretty to start... let's just make the thing I attached pretty, not debate the content of the page.
(In reply to comment #5)
> Created an attachment (id=408443) [details]
> nightly web page contents

> morgamic asked whether it should it be .com or .org... I said "both!"

In terms of visual design, we should pick one and then have the other redirect to it. Meaning, if the URL is nightly.mozilla.org then it should be a page in the mozilla.org theme, and then nightly.mozilla.com could just redirect to that (or vice versa).

Mozilla.com is generally the home of Firefox and Firefox-related content, so nightly.mozilla.com makes sense on that level. But, I know there's a history with mozilla.org here. If we were starting from scratch I'd vote for mozilla.com and that design theme, but I think either could work.
(In reply to comment #9)
> (In reply to comment #5)
> > Created an attachment (id=408443) [details] [details]
> > nightly web page contents
> 
> > morgamic asked whether it should it be .com or .org... I said "both!"
> 
> In terms of visual design, we should pick one and then have the other redirect
> to it. Meaning, if the URL is nightly.mozilla.org then it should be a page in
> the mozilla.org theme, and then nightly.mozilla.com could just redirect to that
> (or vice versa)

Sounds good. A search for "firefox nightly" shows only mozilla.org pages, so I think we should go with that, with the mozilla.com redirect.
Agreed about comment #10.  Unless there's a really good reason, there shouldn't be any sites that are on *.mozilla.com that don't redirect to *.mozilla.org (or vice versa).  As for which of the domains to default to, my personal opinion is that things should default to mozilla.org unless there's a good reason not to.
(In reply to comment #2)
> Actually, this is closer to update-watch. That already scrapes ftp for nightly
> builds. 

OK, so is the way forward here to modify update-watch, or what?
If what you want to do is scrape some timestamps and filenames out of 
  http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/
and generate some html then update-watch is not well suited. That's best done in [your favorite scripting language]. 

Any reason you're linking to the zip and not the installer for windows ? We should promote the build types we ship as releases.
(If we link to the installer, it would be good to add a quick note along the lines of... "Already using a nightly? Use Help -> Check for Updates to download the latest nightly", but we can add that later.)
(In reply to comment #13)
> If what you want to do is scrape some timestamps and filenames out of 
>   http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/

I want the thing that uploads to that directory to generate an HTML page. Are you telling me that's actually a bunch of distinct processes, so scraping is the way to go?
Do you want an HTML page, or some formatted data we can use to generate an HTML page? I'd think the latter is easier for everyone, and then page layout changes don't get tied up with build infra...
Posted file generate the HTML page (obsolete) —
this scrapes the mod_autoindex-generated apache page from ftp.mozilla.org and then builds something like the HTML page I attached.

Needs a sort function to put things in the right order, and maybe some date formatting.
Attachment #408519 - Attachment mime type: text/x-python-script → text/plain
Posted file order the builds (obsolete) —
this uses the order the builds are defined in, avoiding the sorting issue. I'll make the list members a class I guess, once we figure out exactly what's needed.
Attachment #408519 - Attachment is obsolete: true
Attachment #408528 - Attachment is obsolete: true
Posted file sample output
Attachment #408443 - Attachment is obsolete: true
ok, do I need to add a bunch of error handling to this? or can I count on some cron infrastructure to send out alerts if it starts exiting with an error code?

seems like we could install the thing I attached in here and get started setting up the redirects, without advertising that the page exists.
Need to get this into version control somewhere. Maybe a new repo under http://hg.mozilla.org/webtools/?
I bet morgamic will know where to put this file, too.
Attachment #408532 - Attachment is obsolete: true
Attachment #408650 - Flags: review?(morgamic)
Comment on attachment 408650 [details] [diff] [review]
scrape the build server, make a nice HTML page

Thoughts:
- bug 405088 has an idea about creating a generic feed for all product information (not just nightlies); this could be extended to meet that need at some point
- missing doctype, just use 4.0.1 trans or somethin
- style; do we care?
- if we made the output JSON we could just syndicate it and slurp it/inject it anywhere else; any feelings about that?
- patch seems simple enough and can't get xss'd unless someone MITM's the ftp url...

Next:
- add doctype
- consider writing JSOn output and HTML counterpart that eats JSON and spits out HTML output (and that won't have to change via cron, only the JSON will)
- set up the domain
- file IT bug to host it

Overall, I think it's fine as is but we could expand it to be easier to integrate with other sites.  Do you think it's worth doing?
Attachment #408650 - Flags: review?(morgamic) → review+
Regarding my JSON comments in comment #24, just creating a buildJSON(builds) method would work and you could add that to main()...
(In reply to comment #24)
> (From update of attachment 408650 [details] [diff] [review])
> Thoughts:
> - bug 405088 has an idea about creating a generic feed for all product
> information (not just nightlies); this could be extended to meet that need at
> some point

A fair point, but that bug is even older than this one. ;)

> - missing doctype, just use 4.0.1 trans or somethin

I'll add an HTML5 doctype.

> - style; do we care?

Of course we care. I'll file a follow-up for jslater or his delegate to style it. I'll tweak the markup as needed then.

> - if we made the output JSON we could just syndicate it and slurp it/inject it
> anywhere else; any feelings about that?

I can spit out JSON no problem, assuming there's a json module I can use, or just copy yet another copy of simplejson. This means the file would need some config information since it would be spitting out two files. Right now, it can just write to stdout.

> - patch seems simple enough and can't get xss'd unless someone MITM's the ftp
> url...

If it gets fancier, a templating system that actually prevents xss (genshi etc) might be good.

> - set up the domain
> - file IT bug to host it

ok, so three bugs. these two, plus the styling bug.
(In reply to comment #26)
> > - set up the domain
> > - file IT bug to host it

That should just be one bug, not two... Also, I'll repeat comment #22: This needs to be in version control somewhere.
Blocks: 524881
Assignee: nobody → sayrer
dang it, reed is right. hate it when that happens.

where should we put it?
IT: bug 524879
Make sexy: bug 524881

Umm - hg is fine, or svn.  Up to you, you're the author!
Filed bug 524962 to create the repository.
Depends on: 524962
dumps to /tmp by default, makes two files: index.html and index.json
Attachment #408650 - Attachment is obsolete: true
Attachment #408884 - Flags: review?(morgamic)
Comment on attachment 408884 [details]
address review comments

Valid html, JSON looks good.  Hhave to remind IT about the output_dir, simplejson and python 2.6.  Should be easy to add the mozilla.org header/footer crap -- I can do that in the other bug.
Attachment #408884 - Flags: review?(morgamic) → review+
http://hg.mozilla.org/webtools/nightly/rev/4b18c73bd660
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Depends on: 527153
You need to log in before you can comment on or make changes to this bug.