1464327 - geckodriver documentation on firefox-source-docs.mozilla.org is no longer getting updated

Status: RESOLVED FIXED

(Developer Infrastructure :: Source Documentation, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

On bug 1460307 I added documentation in how to retrieve crash report details for automated tests as executed via geckodriver. I pushed the commit on May 11th and it got merged to mozilla-central on May 12th:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=1b2671e3e9f2&filter-searchStr=doc&filter-tier=1&filter-tier=2&filter-tier=3

The DocUp job was running but the newly added documentation never has been uploaded and as such is not available at:

https://firefox-source-docs.mozilla.org/testing/geckodriver/geckodriver/index.html

When I run `mach doc` locally I can perfectly see the docs present and the appropriate HTML page has been generated.

Is something broken with uploads of docs right now?

Comment 1

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Yeah, I also modified some docs under Marionette a few weeks ago and the changes are not showing up either.

Comment 2

[Gregory Szorc [:gps]]

You aren't the first person to notice that docs upload is broken.

I triggered a "docup" task against the latest mozilla-central revision at https://tools.taskcluster.net/groups/MuSx1h4DRVy4257G1J_qmA/tasks/NKki0q7MSaGV2b_ZeJjCXg/runs/0. Let's see if that fixes anything.

A proper fix is to find out why the task isn't running automatically or why it isn't working.

I'm moving the bug component because I feel this has more to do with automation configuration than docs generation.

Comment 3

[Gregory Szorc [:gps]]

Docs are being uploaded properly... to https://firefox-source-docs.mozilla.org/main/latest/.

We can see this in logs like https://taskcluster-artifacts.net/RPNDCL7BS_SmoULeHwmNIw/0/public/logs/live_backing.log.

I can't recall how we have the / URL on that site set up. Were we supposed to redirect / to /main/latest? Did that get messed up?

Comment 4

[Gregory Szorc [:gps]]

From tools/docs/mach_commands.py:

        # Until we redirect / to main/latest, upload the main docs
        # to the root.
        if project == 'main':
            s3_upload(files)

I suspect recent refactorings by ahal may have broken this branch.

ahal: could you please look into this?

Comment 5

[Andrew Halberstadt [:ahal]]

My apologies, looks like I forgot to remove a trailing comma and the project got changed to a tuple, e.g: (u'main',)

The DocUp task is able to successfully upload to "(u'main',)/62.0/_sources/..." on the server, e.g:
https://taskcluster-artifacts.net/D7zjNgBoTGiDF4c2NGv7OQ/0/public/logs/live_backing.log

This explains why this didn't get caught by CI.

Comment 6

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1464327 - [docs] Fix doc uploading by removing trailing comma,

This was changing the project from "main" -> ("main",) which was causing the DocUp
task to upload to urls like:
"('main',)/62.0/_sources/..."

Review commit: https://reviewboard.mozilla.org/r/247228/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/247228/

Comment 7

[Gregory Szorc [:gps]]

Comment on attachment 8981136 [details]
Bug 1464327 - [docs] Fix doc uploading by removing trailing comma,

https://reviewboard.mozilla.org/r/247228/#review253806

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a5b4905bb617
[docs] Fix doc uploading by removing trailing comma, r=gps

Comment 9

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/a5b4905bb617

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So I triggered an upload job via `add-new jobs` and it looks like it was run even not visible in Treeherder:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=c094fc31b95f079c5cec2f3e88cb9fec0bdd7ca5&filter-searchStr=add-new&selectedJob=180856310

Task details are here:

https://tools.taskcluster.net/groups/BgDz92GqQ-S6hHBh96o8tQ/tasks/VmqegpbxQvuafVJ-x8K1oQ/runs/0/logs/public%2Flogs%2Flive.log

It was successful, but still no sign of the CrashReports docs for geckodriver being present:

https://firefox-source-docs.mozilla.org/testing/geckodriver/geckodriver/CrashReports.html

Comment 11

[Andrew Halberstadt [:ahal]]

They're there:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=c094fc31b95f079c5cec2f3e88cb9fec0bdd7ca5&filter-tier=3&filter-searchStr=DocUp

(they're tier 3 and you had 'Add-New' in the search filter)

And are these your docs?
https://firefox-source-docs.mozilla.org/testing/geckodriver/doc/geckodriver/CrashReports.html

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Oh, I missed the Tier-3 part. And yes, those are the docs. Maybe it took a bit longer until they are available or some kind of caching. Anyway it's great that it works. Thanks!

Comment 13

[Dustin J. Mitchell [:dustin] (he/him)]

They are behind a cloudfront CDN so yes, it takes a while for new docs to appear.

Comment 14

[Gregory Szorc [:gps]]

If you hold shift when clicking the refresh icon in Firefox, you'll force sending of the necessary HTTP headers to not use a cache. I /think/ this results in CloudFront validating the request against the S3 origin and always retrieving the latest document version.

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+1]]

That's what I did but which wasn't working. Even not after multiple force refresh. Anyway it works now.

Comment 16

[Dustin J. Mitchell [:dustin] (he/him)]

My experience is that a force-refresh doesn't affect cloudfront.