How to create a new document in Sphinx/docutils by API?

Question

I' writing a new extension for Sphinx as a domain offering multiple directives, roles and indices for the hardware description language VHDL. This extension shall be able to auto document language constructs. In VHDL, we have e.g. entities/architectures or packages/package bodies. These could be documented as pairs in individual docutils documents, so each language construct has an individual URL (or page in PDF) in the documentation.

I'm looking for a solution to create new documents in Sphinx/docutils. According to the docutils API description, a document is the root element to a doctree (documentation tree). According to the Sphinx documentation, directives are items in the doctree that get consumed and can emit new nodes to the surrounding doctree. So it's a translation/replacement approach.

Non of the documentation seams to offer a way to create new document.

Looking at the autodoc extensions, there are 2 sides. There is sphinx.ext.autodoc coming with Sphinx. It offers .. auto*** directives to automatically document Python language constructs. It requires a user to place dozens to hundreds of auto-directives into the ReStructured Text. Of cause it automatically document e.g. classes or modules, but for a huge project it's a lot of work.

In addition, there is autoapi, which reads the Python code for a given package or module and generates ReST files on the fly when Sphinx is loaded. Then these files - containing auto-directives - are processed.

As I understand, autoapi workaround the problem of creating now docutils documents, by writing ReST files, so Sphinx generates document instances with a doctree, and then autodoc from Sphinx jumps in and replaces them with content from doc-strings.

So my questions after all investigations I did so far:

• How can I create docutils or Sphinx document to get a HTML file per item I want to auto document?
• Or do I always need to do a hack like autoapi from Carlos Jenkins and create ReST files as dummies or with auto directives, so I can use the replacement capabilities of Sphinx/autodoc from directive to documentation nodes?

---

Why don't I like the autoapi approach? I have parse VHDL files as input in form of a Code Document Object Model (CodeDOM). I don't want to serialize parse VHDL file into ReST, to parse it again, construct again a model of my source files, so I can then translate to documentation nodes like sections, paragraphs and lists.

I have all available to generate doc-nodes for docutils, but I need multiple documents, so I can distribute the content to hundreds of documentations file (HTML files).

Answer 1

I am in a similar situation, and I am right now looking at generating an html file per source file. The following is a snapshot of where I was at when I found your question:

You can tell sphinx to parse other file types at build time. In conf.py:

source_suffix = {
    '.rst': 'restructuredtext',
    '.something': 'my_language',  # Will call the my_language parser for .something files
}

Then create your own extension (could be part of the VHDL domain you created). Either in my_extension.py or my_extension/{__init__.py,etc}

from docutils import nodes
from sphinx.parsers import Parser

class MyLanguageParser(Parser):
    supported = ("my_language",)  # Declate supported languages
    def parse(self, inputstring, document):
        # Do something more complex, but a document can be created from the file (contents in inputstrings)
        txt = nodes.Text('HI')
        document += txt

def setup(app):  # This is called when loading the extension
    app.add_source_suffix('.something', 'my_language')
    app.add_source_parser(MyLanguageParser)

    return {'version': '0.1'}  # Optional, but nice to version your extension

Now, when calling sphinx-build, such as with make html, your parser will be called and generate an output page per input file.

---

To answer the title of your question, you can also programatically construct new arbitrary documents this way:

from sphinx.util.docutils import new_document
from docutils.core import publish_doctree, publish_from_doctree
from docutils import nodes

# create a new document
doc = new_document('mydoc')
txt = nodes.Text('HI')
doc += txt

# Alternatively, you can also build it from rst:
rst = """
Doc Title
=========

some content
"""
doc = publish_doctree(rst)

with open('mydoc.html', 'wb') as f:
    f.write(publish_from_doctree(doc, writer_name='html'))

The last part is taken from here.