How to create new documents with a sphinx extension

Question

From a sphinx extension, how do I create new documents (new page) under the current document that called my extension. I.e, I have an extension that is called from a rst file.

Something like:

foo.rst
  .. myextension::

During myextension, I want to generate new files under foo. Something as:

foo/bar.html:
   * Some documentation in a new page
foo/baz.html:
   * More documentation in a new page

This is a simplification of my extension, if that helps:

class NetYnlInclude(Directive):
    def run(self):
        file = "my_file.txt"     
        self.state.document.settings.record_dependencies.add(file)                     
        parsed_str = parse(file) # Parses the file into a string
        self.state_machine.insert_input(
            statemachine.string2lines(parsed_str), filename)

        paragraph_node = nodes.paragraph()
        return paragraph_node

Answer 1

If you want to generate .html files directly I'm not sure how to integrate that with Sphinx.

If you want to generate .rst files and you want your new documents to be processed by Sphinx you need to create them at a very early stage in the Sphinx build pipeline.

As far as I understand attaching to the build-inited event like autosummary does is the way to do it.

So your extension needs something like

def setup(app: Sphinx) -> None:
    app.connect('builder-inited', generate)


def generate(app: Sphinx) -> None:
    # write .rst files here

Now notice there's no mention of directives anywhere - generate has no access to any parsed .rst documents, directives etc. because at this point no documented have been parsed yet. If you want to extract information from .rst files you need to do parse them manually (which is also what `autosummary does).

Alternatively your extension could accept some configuration options to tell it what files to ingest and what to generate.

For a more complete answer see this nice blog post and a corresponding complete, self-contained repository with the code.