How to Set Up a Sphinx Autodoc Pipeline with Markdown and Mermaid

This document explains how to set up a reliable Sphinx autodoc pipeline for a Python codebase using:

autodoc and autosummary for API documentation
Markdown (.md) files via MyST
Mermaid.js diagrams rendered during the Sphinx build
A clean separation between source code, documentation source, and build output

The goal is a pipeline that is:

Deterministic
Reproducible
Import-safe
Suitable for long-lived projects

Conceptual Overview

The pipeline has three responsibilities:

Import Python code safely
Extract documentation from docstrings
Combine generated API docs with authored Markdown content

Sphinx is used as the orchestrator, not as a content author.

Required Project Structure

A minimal, sane layout looks like this:

my_project/
├── src/
│   └── my_project/
│       ├── __init__.py
│       ├── core.py
│       └── utils.py
├── docs/
│   ├── conf.py
│   ├── index.md
│   ├── api/
│   │   └── index.md
│   ├── diagrams/
│   │   └── architecture.md
│   └── _static/
├── pyproject.toml
└── README.md

Key rules:

Code lives under src/
Docs live under docs/
Sphinx never mutates source code
Documentation output is disposable

Step 1: Create a Virtual Environment

From the project root:

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip

This environment must be active whenever you build docs.

Step 2: Install Dependencies

Install Sphinx and required extensions:

python -m pip install \
  sphinx \
  sphinx-rtd-theme \
  myst-parser \
  sphinxcontrib-mermaid

Optionally freeze them later in requirements.txt.

Step 3: Initialize the Sphinx Project

From the project root:

cd docs
sphinx-quickstart

Use these answers:

Separate source and build directories: No
Project name: your project name
Author: your name
Language: en

This creates:

conf.py
index.rst (you will replace this)

Delete index.rst.

Step 4: Configure `conf.py`

Make Code Importable

At the top of conf.py:

import os
import sys

sys.path.insert(0, os.path.abspath("../src"))

This allows autodoc to import your package.

Enable Extensions

extensions = [
    "sphinx.ext.autodoc",
    "sphinx.ext.autosummary",
    "sphinx.ext.napoleon",
    "sphinx.ext.viewcode",
    "myst_parser",
    "sphinxcontrib.mermaid",
]

Enable autosummary generation:

autosummary_generate = True

Configure MyST Markdown

myst_enable_extensions = [
    "colon_fence",
    "deflist",
    "fieldlist",
]

Theme and Static Assets

html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]

Step 5: Replace `index.md`

Create docs/index.md:

# My Project Documentation

This site contains developer documentation for `my_project`.

## User and Conceptual Documentation

```{toctree}
:maxdepth: 2

diagrams/architecture

API Reference

This file is the root of the documentation graph.

Step 6: Set Up API Autodoc Pages

Create API Index

docs/api/index.md:

# API Reference

```{toctree}
:maxdepth: 2

modules

### Create Autosummary Stub

`docs/api/modules.md`:

```md
/``{autosummary}
:toctree: generated
:recursive:

my_project
/``

This tells Sphinx to:

Import my_project
Generate per-module API pages
Place them under docs/api/generated/

You do not write those files by hand.

Step 7: Write Good Docstrings

Sphinx autodoc is only as good as your docstrings.

Example:

def parse_config(path: str) -> dict:
    """
    Parse a configuration file.

    Args:
        path: Path to the configuration file.

    Returns:
        Parsed configuration as a dictionary.
    """

Use:

Google-style or NumPy-style docstrings
No side effects at import time
No runtime dependencies in module top-level code

Step 8: Add Mermaid Diagrams

Create a Markdown file:

docs/diagrams/architecture.md

Architecture

graph TD
    A[User] --> B[CLI]
    B --> C[Core Engine]
    C --> D[Output]

Sphinx will render this during the build.

Step 9: Build the Documentation

From the docs/ directory:

python -m sphinx -b html . _build/html

Output appears in:

docs/_build/html/

Open index.html in a browser to verify.

Step 10: Iterate Safely

When modifying the pipeline:

Change one thing at a time
Rebuild often
Treat _build/ as disposable
Never commit generated API files unless publishing artifacts

Common Failure Modes

Code imports fail because dependencies are missing
Side effects execute at import time
Modules rely on runtime configuration
Autosummary files are edited by hand
Markdown files are not added to a toctree

All of these are fixable, but avoidable.

Recommended Practices

Keep docstrings import-safe
Treat documentation as a build artifact
Separate authored docs from generated docs
Document the pipeline itself
Automate builds only after manual success

Summary

A Sphinx autodoc pipeline works when:

Python code is importable
Docstrings are authoritative
Markdown provides narrative structure
Mermaid diagrams illustrate architecture
Sphinx assembles everything deterministically

Once set up correctly, the system scales without surprises.