1557020 - The watch functionality in ./mach doc loses the document tree

Status: RESOLVED FIXED

(Developer Infrastructure :: Source Documentation, defect, P3)

Description

[Mark Banner (:standard8)]

When I run ./mach doc and then edit a document, the document is correctly refreshed, however the entire document tree for all of mozilla-central is lost, and only the page I just changed is present.

On the output, I see (amongst other things):

copying static files... WARNING: html_static_path entry u'/Users/mark/dev/gecko/tools/docs/_static' does not exist

Comment 1

[Andrew Halberstadt [:ahal]]

Yes this is rather unfortunate, and something I tried to fix for quite a long time when I initially implemented the livereload. But I ultimately ended up giving up (I don't remember the exact specifics of why it was so hard to fix). But I suspect fixing this might require a substantial re-write of the doc system.

Comment 3

[Akshat [:baka]]

After a lot of reading code and debugging I could not find much. But I did discover something interesting, If I just open the index.html file after it is rendered and save it. No changes done whole website somehow corrects itself. :ahal Can you point me in the right direction since you have already looked into this bug. Any information is helpful.

Comment 4

[Andrew Halberstadt [:ahal]]

Sorry, my memory here is really foggy. When I wrote comment 1 it had already been years since I looked into this, so it's likely been at least 4 years since I investigated.

I vaguely recall that I could get it working, but then it caused some other problem (that was an even bigger deal), and it was solving both issues at the same time that was really difficult.

I assume you found this function, but in case not it will likely need some heavy changes as part of this bug:
https://searchfox.org/mozilla-central/source/tools/moztreedocs/__init__.py#153

Also this is definitely not an easy one to pick up. If you're confident and up for a challenge then you are more than welcome to keep digging. But if you are looking for a good bug to practice contributing to Mozilla, this isn't it :).

Comment 5

[Akshat [:baka]]

Thankyou :ahal for that information. I have found this function, and I tried to debug and saw the tree built to remain unchanges and even when the wrong index is loaded the and if you click on view source code. The correct tree displays there. Can you comment about the interesting thing I mentioned?

" If I just open the index.html file after it is rendered and save it."

This is from the place where moztreedocs keep html files. ( I have contributed to a few bugs, even a good one and I am trying to take the mozdocs project for this outreachy round so this bug will help my progress a lot and according to :sylvestre this one is annoying as well! )

Comment 6

[Andrew Halberstadt [:ahal]]

Yes, it is indeed an annoying one!

Thanks for the extra context, I was going to help Sylvestre co-mentor that project and I agree this would be a nice one to fix. I dug around in the code some more and I recall a few extra details:

1. I think the "other problem" I was trying to solve was only rebuilding the modified doc tree. Because generating the entire tree takes so long, it's kind of useless to use in the livereload server (the whole point of that is to get rapid feedback). So I was trying to make modifying a file only rebuild the sub-tree it is part of, while also preserving the rest of the sub-trees that weren't modified. At the time I chose to prioritize rebuilding fast at the expense of losing the index.html. We could likely switch this around if folks disagree with my choice, though IMO the livereload server is even more useless in that case.

2. I think part of the reason this is hard is because the index.rst is auto-generated based on the sub-trees that are passed in:
   https://searchfox.org/mozilla-central/source/docs/index.rst
   https://searchfox.org/mozilla-central/source/tools/moztreedocs/__init__.py#226

3. As for your interesting observation, when you modify the root index, the doc system is likely picking up all sub-trees again and therefore doing a full generation (which causes the index in 2. to be populated with the full set of docs).

Comment 7

[ogiorgis]

Attachment: Bug 1557020 - Don't remove unaccounted files while building docs r=sylvestre

While copying sphinx files, some unmodified static files were deleted.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c0f42e0c457a
Don't remove unaccounted files while building docs r=sylvestre

Comment 9

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/c0f42e0c457a