Documentation as Infrastructure: Scaling Knowledge Across Engineering Teams

Documentation debt slows engineering teams in ways that look fine until the wrong person leaves. The cost surfaces during a payment incident, a migration, or an audit: the written procedure exists but the deep system understanding doesn’t. The documentation that survives that moment has three layers: decisions captured as RFCs and ADRs, system reference generated from the systems themselves, and process docs written from real examples. Everything else is tooling and the review habits that keep those three layers current.

Common Documentation Patterns

Some recurring challenges in documentation appear across many organizations:

Level 1: The Wiki Graveyard

• Thousands of pages in Confluence
• Most of them outdated or irrelevant
• Search returns hundreds of results for “authentication”
• Nobody knows which one is current

Level 2: README Roulette

• Every repository has different documentation standards
• Quality varies from excellent to non-existent
• New engineers play guessing games about which README to trust

Level 3: Slack Knowledge

• Critical architectural decisions buried in #general
• “Remember that conversation about the database migration?” No, nobody does
• Institutional knowledge trapped in private DMs

Level 4: Hero Documentation

• One person knows everything about the billing system
• They’re overloaded with questions
• When they leave, knowledge walks out the door

Level 5: Meeting Minutes Maze

• Important decisions scattered across hundreds of Google Docs
• No consistent format or structure
• Finding the rationale for a design choice requires archaeological skills

If any of these sound familiar, you’re not alone. The tool in use matters far less than how a team thinks about information architecture. Adding more pages to Confluence or Notion won’t fix it; without ownership and a review habit, even a well-structured wiki goes stale.

Documentation Debt vs Technical Debt

We spend a lot of time discussing technical debt, but documentation debt can be even trickier to spot. Technical debt usually shows up in slower deployments or harder maintenance. Documentation debt shows up when teams start second-guessing decisions they made six months ago because no one remembers the reasoning.

The cost lands in three places. Day to day, it lands on onboarding time, on the interruptions senior engineers absorb, and on two teams solving the same problem without knowing it. Underneath that sit the quieter costs: repeating mistakes the organization already made, and hesitating to change systems nobody can explain. In a crisis it becomes an incident that runs long because the person who understands the system is unreachable, or an audit where nobody can reconstruct why a control was designed that way.

Teams who view documentation as “extra work” often end up spending more time later explaining, re-explaining, and re-discovering the same information.

A Three-Layer Documentation Approach

In practice, a three-layer approach scales reasonably well:

Layer 1: Decision Architecture (The Why)

This is where you capture the reasoning behind choices. Not what you built, but why you built it that way.

/docs
  /decisions  # ADRs - architecture decisions made
  /proposals  # RFCs - future changes being considered  
  /discussions  # RFDs - open problems being explored

Template approach that works well:

Mini-RFC (1-2 pages):

• Single team impact
• Reversible decisions
• 1-week timeline

Standard RFC (5-10 pages):

• Multi-team impact
• Significant investment
• 2-4 week timeline

Strategic RFC (10+ pages):

• Company-wide impact
• Major architectural changes
• 6+ week timeline

Layer 2: System Documentation (The What)

This describes your current reality. What exists, how it connects, who owns it.

/systems
  /service-catalog  # What services exist, who owns them
  /architecture  # How systems connect and communicate
  /runbooks  # How to operate and troubleshoot
  /dependencies  # What depends on what

Important insight: This layer works best when it’s mostly automated. Hand-written system docs seem to become outdated the moment you finish writing them. Documentation derived from OpenAPI schemas, Terraform state, or CI pipelines updates alongside the change that made it stale.

Layer 3: Process Documentation (The How)

This captures your cultural DNA. How you work, how you make decisions, how you handle incidents.

/processes
  /engineering  # How we design, build, and review
  /oncall  # How we respond to incidents
  /releases  # How we deploy and rollback
  /hiring  # How we evaluate and onboard

Useful pattern: Process docs tend to work better with concrete examples rather than just abstract guidelines. Engineers learn better from “here’s what was actually done” rather than “here’s what should be done.”

The Amazon vs Google Documentation Philosophy

Two main approaches come up often across larger organizations:

Amazon’s Narrative Approach

6-page written narratives instead of PowerPoint presentations:

• Forces complete thinking before meetings
• Creates artifact of the decision process
• “Study hall” format ensures everyone actually reads

Adapted structure (results vary by team):

1. Executive Summary (1 page)
2. Context and Problem (1 page)
3. Proposed Solution (2 pages)
4. Alternatives Considered (1 page)
5. Implementation Plan (1 page)
6. Appendix (unlimited)

Google’s Design Doc Culture

Collaborative technical documents with peer review:

• Emphasis on trade-offs and alternatives
• System context diagrams
• Async collaboration through comments

Key elements:

• Context and Scope - What are we solving?
• Goals and Non-Goals - What success looks like
• Design - How we’ll solve it
• Alternatives - What we considered and rejected
• Cross-cutting Concerns - Security, performance, monitoring

A useful hybrid: Combining Amazon’s “force yourself to think it through” approach with Google’s collaborative review culture. Different team dynamics respond better to different approaches, so results vary.

Documentation as Code: The Technical Implementation

Treat documentation like any other critical infrastructure:

# .github/workflows/docs.yml
name: Documentation Infrastructure
on:
  pull_request:
    paths: ['docs/**', 'adr/**', 'rfcs/**']

jobs:
  validate-documentation:
    runs-on: ubuntu-latest
    steps:
      - name: Validate RFC format
        run: |
          # Check required sections exist
          # Validate YAML frontmatter
          # Ensure decision status is valid
      
      - name: Check broken links
        run: |
          # Scan for dead internal links
          # Verify external links return 200
          # Flag links to deprecated services
          
      - name: Generate architecture diagrams
        run: |
          # Auto-generate from PlantUML source
          # Update system dependency graphs
          # Create visual service maps
          
      - name: Update search index
        run: |
          # Index new content for searchability
          # Tag documents with metadata
          # Update recommendation engine

Tool stack that works well in practice:

• MkDocs Material - Beautiful, searchable documentation sites
• PlantUML/Mermaid - Version-controlled architecture diagrams
• ADR-tools - Command-line decision record management
• GitHub Actions - Automated validation and publishing

The DACI Framework for Documentation Decisions

For any significant technical decision, the DACI framework (Driver, Approver, Contributors, Informed) keeps the roles clear:

# RFC-042: Database Migration Strategy

## DACI Matrix
- **Driver:** Database Team Lead
  - Responsible for gathering input and driving to decision
  - Owns the timeline and process
  
- **Approver:** VP Engineering  
  - Makes the final call
  - Accountable for the outcome
  
- **Contributors:** Backend Teams, SRE, Security, Data Engineering
  - Provide input and expertise
  - Will be impacted by the decision
  
- **Informed:** All Engineering, Product, Finance
  - Need to know the outcome
  - May need to adjust their plans

## Decision Timeline
- **Week 1:** Stakeholder interviews and requirements gathering
- **Week 2:** Technical evaluation and proof of concepts
- **Week 3:** Cost analysis and migration planning
- **Week 4:** Final decision and communication

This framework helps avoid the “too many cooks” situation while still making sure people feel heard. Getting the balance right takes some trial and error.

Scaling Documentation Culture: The Champion Network

Documentation culture can’t really be mandated from above - it tends to work better when it grows more naturally. But conditions can be created that make it more likely to take root.

The Documentation Champion Approach

One approach that can work is having a “Documentation Champion” per team (typically one champion for every 5-8 engineers):

Responsibilities:

• Facilitate RFC reviews within their team
• Ensure new systems come with proper documentation
• Identify knowledge gaps and outdated information
• Coach team members on documentation standards

Time commitment: ~2 hours per week Rotation: Every 6 months to prevent burnout

Documentation Metrics Worth Tracking

Many teams track things that don’t necessarily correlate with documentation health. Here is what tends to be more useful to measure:

interface DocumentationHealth {
  // Leading indicators (predict future problems)
  rfcParticipation: number;  // % engineers participating in RFC reviews
  docUpdateFrequency: number;  // Average days since last update
  knowledgeDistribution: number;  // % of systems with >1 expert
  
  // Lagging indicators (measure current state)
  onboardingVelocity: number;  // Days from hire to first commit
  crossTeamQuestions: number;  // Questions requiring cross-team knowledge
  
  // Quality indicators (measure documentation value)
  documentRelevance: number;  // % of docs accessed in last 90 days
  linkHealth: number;  // % of internal links that work
  searchSuccess: number;  // % of searches that find answers
}

Monthly review questions:

1. Which knowledge gaps caused delays this month?
2. What questions were asked multiple times?
3. Which documents are becoming stale?
4. Where are people going outside our documentation system?

Where Documentation Changes the Outcome

Three situations turn documentation from a nice-to-have into the thing that decides how the day goes.

Incident recovery. A rollback procedure is only useful if someone who has never run it can follow it under pressure. Runbooks that get rehearsed and updated let the on-call engineer restore a system without paging the person who built it. That matters most exactly when that person is unreachable.

Integration after an acquisition or reorg. Most of the time in an integration goes to reconstructing why the other side built things the way they did. When that reasoning already exists as ADRs and design docs, the work shifts from archaeology to migration.

Compliance audits. Auditors ask why access controls and data handling were designed the way they are, and the current configuration alone does not answer that. A trail of ADRs does. Without one, the team reconstructs its own reasoning from memory while the audit clock runs.

Documentation Tools: Which One for What?

Different tools work well in different situations. Your team’s needs might vary, but here are some observations:

Confluence: The Enterprise Classic

When it works:

• Jira integration is critical
• Corporate compliance requires it
• Non-technical stakeholders need access

How to use it properly:

/spaces
  /ENG  # Engineering space
    /RFC  # Templated page tree for RFCs
    /ADR  # Date-based ADR archive
    /Runbooks  # Categorized operation docs
  /PRODUCT  # Product space (for PRDs)

Pro tip: Add dates to Confluence page titles: [2024-01-22] Database Migration RFC. While search has improved significantly, chronological ordering still helps with navigation.

Anti-patterns:

• Putting everything in one space (search hell)
• Not using templates (inconsistent formats)
• Not deleting old pages (use archive labels)

Notion: Modern and Flexible

When it shines:

• You want to use database views
• RFC tracking in Kanban boards
• Rich media and embeds for documentation

Database-based setup:

// RFC Database structure
interface NotionRFC {
  title: string;
  status: 'Draft' | 'Review' | 'Approved' | 'Rejected';
  author: Person;
  reviewers: Person[];
  impactedTeams: MultiSelect;
  decisionDate: Date;
  tags: MultiSelect;
}

Strengths:

• Different views (Table, Board, Timeline, Calendar)
• Rich template system
• AI integration (automatic summarization)
• Version history and collaboration

GitBook: Developer-First Approach

Where it excels:

• Open source projects
• API documentation
• Version-controlled documentation

Git integration:

# .gitbook.yaml
root: ./docs/
structure:
  readme: README.md
  summary: SUMMARY.md
redirects:
  previous/page: new-folder/new-page.md

Advantages:

• GitHub/GitLab sync
• Markdown native
• Can go through code review
• Different versions per branch

Obsidian: Knowledge Graph Approach

When to use:

• Building interconnected knowledge networks
• Personal knowledge management
• Zettelkasten methodology

Enterprise usage:

[[2024-01-22-database-migration]]
Related: [[postgres-best-practices]] | [[migration-checklist]]
Tags: #rfc #database #approved

Power of graph view: Visually shows which systems are related to each other.

SharePoint/Teams Wiki: Microsoft Ecosystem

When it’s mandatory:

• Organizations using Microsoft 365
• Security policies block 3rd party tools
• IT department won’t allow anything else

Best practices:

/sites/Engineering
  /Shared Documents
    /Architecture
      /ADR
        /2024
          01-use-kubernetes.md
          02-migrate-to-postgres.md
    /Processes
      /RFC-Template.docx

Survival tactics:

• Don’t use OneNote as a wiki (search is unreliable)
• Use checkout/checkin for version control
• Set up approval workflows with Power Automate

GitHub/GitLab Wiki: Code-Adjacent Documentation

Ideal usage:

• Repository-specific documentation
• Contributing guidelines
• Development setup

Structure:

.wiki/
  Home.md
  Architecture/
    Decision-Records.md
    System-Overview.md
  Operations/
    Deployment.md
    Rollback.md

Backstage: Developer Portal

For enterprise scale:

• Service catalog
• API documentation
• Tech radar
• Cost tracking

catalog-info.yaml:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: payment-service
  description: Handles payment processing
  annotations:
    docs: https://docs.internal/payment
    pagerduty: PD123
spec:
  type: service
  owner: platform-team
  lifecycle: production

Tool Selection Matrix

Use Case	First Choice	Alternative	Avoid
Engineering RFCs	GitHub + MkDocs	GitBook	SharePoint
Product Documentation	Notion	Confluence	Word Docs
API Docs	GitBook	Backstage	Wiki
Runbooks	MkDocs	Confluence	OneNote
Knowledge Base	Obsidian	Notion	Folders
Service Catalog	Backstage	Custom	Excel

Migration Strategy

From Confluence to MkDocs:

# 1. Export Confluence space
confluence-export --space ENG --format markdown

# 2. Transform to MkDocs structure
python transform_confluence.py --input export/ --output docs/

# 3. Setup redirects for old URLs
# mkdocs.yml
plugins:
  - redirects:
      redirect_maps:
        'old-page.md': 'new-structure/page.md'

Hybrid Approach (Most Common in Practice)

Most organizations use multiple tools:

documentation_stack:
  decisions:
    tool: GitHub + ADR-tools
    reason: "Version control and code review"
  
  product_specs:
    tool: Notion
    reason: "Easy for PMs, rich formats"
  
  runbooks:
    tool: Confluence
    reason: "On-call engineers are familiar"
  
  api_docs:
    tool: GitBook
    reason: "Auto-sync with OpenAPI specs"
  
  knowledge_base:
    tool: Obsidian
    reason: "Connected knowledge graph"

Worth noting: It helps to be clear about where different types of documentation live. When someone asks “Where’s the RFC?” there should ideally be one obvious answer, not a treasure hunt across multiple systems.

Implementation Roadmap

Phase 1: Foundation (Months 1-2)

Week 1-2: Infrastructure Setup

• Deploy MkDocs with search
• Create RFC/ADR templates
• Set up automated validation pipeline
• Establish document approval workflow

Week 3-4: Champion Training

• Select documentation champions
• Train on templates and processes
• Set up regular review cadence
• Create feedback mechanisms

Week 5-8: Pilot Team

• Choose 1-2 teams for pilot
• Migrate critical knowledge
• Run first RFC reviews
• Gather feedback and iterate

Phase 2: Adoption (Months 3-6)

Month 3: Mandate and Standards

• Require RFCs for architectural changes
• No new services without documentation
• Weekly RFC review meetings
• Documentation review in code review

Month 4-5: Knowledge Migration

• Audit existing critical knowledge
• Prioritize based on risk and impact
• Systematic migration to new format
• Retire old documentation systems

Month 6: Culture Integration

• Documentation goals in performance reviews
• Recognition for good documentation
• Documentation debt in planning
• Cross-team RFC participation

Phase 3: Optimization (Months 6-12)

Month 7-9: Automation

• Auto-generate system documentation
• Intelligent document recommendations
• Broken link detection and fixing
• Search analytics and improvement

Month 10-12: Scaling

• Roll out to entire engineering organization
• Advanced analytics and metrics
• Integration with other systems (Slack, JIRA, etc.)
• Continuous improvement processes

Core Documentation Principles

A few principles tend to guide good documentation decisions across different team contexts:

1. Documentation Is a Time Investment

Time spent on solid documentation tends to pay back in multiples. When someone writes a clear ADR, it often prevents the team from having the same architectural debate multiple times over the following months.

2. Consistency Usually Trumps Creativity

Consistent templates and processes tend to scale better than letting everyone find their own approach. When documents follow similar patterns, it’s much easier for people to find information across different teams and projects.

3. Context Often Matters More Than Implementation Details

Code shows you what’s happening, comments explain how, but decision documents capture why. The “why” is usually what survives refactoring, migrations, and rewrites - it’s the institutional memory that’s hardest to reconstruct later.

4. Updated Documents Beat Perfect Documents

A decent document that gets updated regularly beats a perfect document that becomes stale. Building processes that make it easy to keep things current is more valuable than trying to get everything right the first time.

5. Measure How Documentation Gets Used

Instead of counting documents written, look at outcomes: how quickly new team members get productive, whether people can find answers to common questions, how often the same concepts require re-explanation. Accessible knowledge is the goal, and page count is a poor proxy for it.

Getting Started

There’s no need to overhaul everything at once. Starting with something small but visible works well:

This Week:

• Pick one critical system that caused recent confusion
• Write a simple 1-page ADR explaining one architectural decision
• Share it in your team channel and ask for feedback

This Month:

• Create a basic RFC template for your team
• Set up a simple documentation site (even a GitHub wiki works)
• Establish a weekly 30-minute “documentation review” in your team meeting

This Quarter:

• Train 2-3 documentation champions
• Require RFCs for all significant changes
• Measure onboarding time and cross-team questions

When This Approach Pays Off

Documentation-as-infrastructure pays off when knowledge turnover is high, systems are complex enough that no single person holds the full picture, or onboarding friction regularly slows delivery. It is a poor fit for single-engineer projects or short-lived prototypes where the cost of maintenance outweighs the benefit. Start with one ADR for the decision that caused the most confusion this quarter; that single document is enough to test whether the practice fits your team.

References

• Diátaxis Framework - A systematic approach to technical documentation that distinguishes tutorials, how-to guides, reference, and explanation by user need
• Software Documentation Guide - Write the Docs - Community-maintained guide covering documentation process, tooling, and best practices from practitioners
• Architectural Decision Records - Reference hub for ADR formats, templates (including MADR), and tooling for capturing and tracking architectural decisions
• Documentation Best Practices - Google Style Guides - Google’s engineering documentation guidelines, including docs-as-code principles and freshness practices
• Software Engineering at Google - Documentation Chapter - In-depth chapter from the Google SWE book on how documentation is treated as code at scale