User Knowledge Server¶

Goal¶

A code review swarm that doesn't know your codebase gives generic advice. One that can query your architecture doc, search your Javadocs, and look up your coding standards gives actionable advice.

The user has knowledge sitting on disk — architecture docs, API docs, coding standards, the codebase itself. The authoring agent should wire this into the workspace at creation time so agents can query it during execution. The user says "my code is at /projects/myapp and our style guide is STYLE.md" and the authoring agent generates a knowledge MCP server, skills, and workspace config.

This is bootstrap-time knowledge wiring — the initial setup during swarmkit init or swarmkit author knowledge-server. Ongoing maintenance (staleness detection, re-indexing, scheduled refresh) is the Knowledge Curator topology's job (design/details/knowledge-curator.md).

Non-goals¶

Ongoing curation. The Knowledge Curator topology handles freshness validation, stale-entry marking, and scheduled re-indexing. This feature creates the initial server; the curator maintains it.
Universal document understanding. v1 handles text files (code, markdown, JSON, YAML, XML, Javadoc HTML). PDF/DOCX ingestion is a tier 3 enhancement via Kreuzberg.
Hosted search infrastructure. v1 runs entirely in-process with no external services. Qdrant is an optional upgrade path.

Use cases¶

Code review swarm for an existing Java project¶

User: "I want a code review swarm for my Spring Boot app at /projects/myapp. Javadocs are at docs/api/, architecture doc is docs/ARCHITECTURE.md, and we follow Google Java Style."

Authoring agent generates: - A knowledge MCP server that indexes /projects/myapp/src/, docs/api/, docs/ARCHITECTURE.md - Skills: search-codebase, query-api-docs, lookup-coding-standards - Workspace config wiring the server under mcp_servers - Agents assigned the appropriate skills

Code generation agent with API context¶

User: "I need agents that generate code against our REST API. OpenAPI spec is at api/openapi.yaml, and there are 200+ endpoint examples in tests/api/."

Authoring agent generates a knowledge server indexing the OpenAPI spec and test examples. The code generation agent calls search-api-spec before writing code.

Documentation-aware support agent¶

User: "Build a support agent that answers questions about our product. Docs are in docs/ — markdown files, about 500 pages total."

Knowledge server indexes the docs directory. Agent calls search-docs to find relevant sections before composing answers.

Architecture¶

User's existing knowledge (code, docs, API specs)
    ↓
swarmkit author knowledge-server (or swarmkit init)
    ↓
Generated FastMCP server (knowledge_server.py)
    ├── indexes sources at startup (in-memory)
    ├── search(query) → ranked results with file + section + snippet
    ├── get_file(path) → file contents
    └── list_sources() → indexed source catalogue
    ↓
workspace.yaml: mcp_servers entry + skills + archetype assignments
    ↓
swarmkit run: agents query knowledge during execution

Generated server shape¶

The authoring agent generates a knowledge_server.py specific to the user's knowledge sources. Not a generic server with config — a concrete, readable Python file the user can inspect, modify, and version-control.

"""Knowledge server for the myapp code review swarm.

Generated by `swarmkit author knowledge-server`.
Indexes: src/, docs/api/, docs/ARCHITECTURE.md
"""

from __future__ import annotations

import os
from pathlib import Path

from mcp.server.fastmcp import FastMCP

server = FastMCP("myapp-knowledge")

SOURCES = [
    {"path": "src/", "patterns": ["**/*.java"], "label": "source code"},
    {"path": "docs/api/", "patterns": ["**/*.html"], "label": "javadocs"},
    {"path": "docs/ARCHITECTURE.md", "patterns": None, "label": "architecture"},
]

# Index built at startup — in-memory, no external dependencies
_index: dict[str, list[dict]] = {}


def _build_index(root: Path) -> None:
    # ... reads files, splits into sections, builds term-frequency index
    pass


@server.tool()
def search(query: str, max_results: int = 10) -> list[dict]:
    """Search indexed knowledge sources."""
    # ... term-frequency search over _index
    pass


@server.tool()
def get_file(path: str) -> str:
    """Read a specific file from the indexed sources."""
    # ... bounds-checked file read
    pass


@server.tool()
def list_sources() -> list[dict]:
    """List all indexed knowledge sources with file counts."""
    pass


if __name__ == "__main__":
    root = Path(os.environ.get("KNOWLEDGE_ROOT", "."))
    _build_index(root)
    server.run()

The generated server is self-contained. No SwarmKit runtime dependency at execution time — it's just a Python MCP server using the mcp SDK.

Three tiers of search quality¶

Tier	Approach	Corpus size	Dependencies	When to use
1	Keyword search (term frequency)	< 5 MB	None	Small projects, quick start
2	TF-IDF with in-memory index	< 50 MB	scikit-learn (optional)	Medium projects, Javadocs
3	Vector embeddings + Qdrant	Unlimited	Qdrant server, embedding model	Large codebases, enterprise

The authoring agent picks the tier based on the corpus size the user describes. Tier 1 is the default — no questions about infrastructure. Tier 3 is offered when the user mentions large codebases or when file counts exceed practical keyword-search limits.

Tier upgrade path: the generated server can be swapped for a Kreuzberg + Qdrant setup without changing the skill YAMLs or workspace config. The MCP tool interface (search, get_file, list_sources) stays the same. Only the server implementation changes.

Authoring flow¶

`swarmkit author knowledge-server`¶

Dedicated authoring mode. Conversation flow:

"What knowledge do your agents need access to?"
"Where is it? (directories, files, URLs)"
Validate paths exist; warn if directory is large
"What format? (code, markdown, Javadoc HTML, OpenAPI YAML/JSON)"
Auto-detect from file extensions when possible
"What should the server be called?" (default: <workspace-id>-knowledge)
Generate:
knowledge_server.py in the workspace directory
Skill YAMLs (search-<label>, get-<label>-file)
mcp_servers entry in workspace.yaml
Validate the workspace resolves cleanly
Optionally test: launch the server, call list_sources, confirm file counts match expectations

Integration with `swarmkit init`¶

During workspace creation, after the user describes their use case:

If the description mentions existing code, docs, APIs, or knowledge sources → proactively ask: "Do you have existing documentation or code the agents should understand? I can set up a knowledge server so they can search it during execution."
If yes → run the knowledge-server authoring flow inline
If no → skip (agents work with LLM knowledge only)

Workspace wiring¶

Generated workspace.yaml additions:

mcp_servers:
  - id: myapp-knowledge
    transport: stdio
    command: ["python", "knowledge_server.py"]
    env:
      KNOWLEDGE_ROOT: "/projects/myapp"

Generated skill:

apiVersion: swarmkit/v1
kind: Skill
metadata:
  id: search-codebase
  name: Search Codebase
  description: >
    Searches the myapp codebase — source files, API docs, and
    architecture documentation. Returns ranked results with file
    path, section heading, and context snippet.
category: capability
implementation:
  type: mcp_tool
  server: myapp-knowledge
  tool: search
iam:
  required_scopes: [knowledge:read]
provenance:
  authored_by: authored_by_swarm
  version: 1.0.0

Relation to other knowledge features¶

                    Bootstrap                    Ongoing maintenance
                    ─────────                    ───────────────────
User docs/code  →  User Knowledge Server    →   Knowledge Curator topology
                   (this feature)                (design/details/knowledge-curator.md)
                   Generated at authoring        Runs on schedule
                   time. Static index.           Validates freshness,
                   No external deps.             re-indexes changes.

SwarmKit docs   →  SwarmKit Knowledge MCP   →   (updated by repo commits)
                   (design/details/knowledge-mcp-server.md)
                   Built into the framework.
                   Exposes design docs +
                   schemas + examples.

The User Knowledge Server bridges the gap between "I have docs" and "my agents can query them." The Knowledge Curator picks up from there for ongoing maintenance. They share the same MCP tool interface (search, get_file, list_sources) so upgrading from the bootstrap server to curator-managed Qdrant is a server swap, not a skill rewrite.

Security¶

Path traversal. The get_file tool must bounds-check paths against the configured source directories. No ../../etc/passwd.
Secrets in docs. The authoring agent should warn if indexed directories contain .env, credentials.json, or files matching common secret patterns. The generated server can exclude patterns via a --exclude list.
IAM scopes. Skills require knowledge:read. In AGT mode, agents must be granted this scope. Workers get read-only; only the Knowledge Curator (if deployed) gets knowledge:write.

Test plan¶

Authoring flow test: scripted conversation with the authoring agent, providing a test directory with known files. Verify the generated server, skills, and workspace config are valid.
Generated server test: launch the generated server via stdio_client. Call search, get_file, list_sources. Verify results match the indexed files.
Live pipeline test: run a topology that uses the knowledge server against a real provider. Verify the agent calls search and incorporates the result into its output.
Security test: attempt path traversal via get_file("../../etc/passwd"). Verify rejection.

Implementation plan¶

PR 1: This design note¶

Design review before implementation.

PR 2: Knowledge server generator template¶

packages/runtime/src/swarmkit_runtime/knowledge/_user_server_template.py — the parameterized template the authoring agent fills in
Tier 1 keyword search implementation
Unit tests: generate → launch → query → verify

PR 3: `swarmkit author knowledge-server` authoring mode¶

New authoring mode in _prompts.py with the conversation flow
Authoring agent generates server + skills + workspace config
Integration with swarmkit init (proactive knowledge question)
Live pipeline test

PR 4 (optional): Tier 2 TF-IDF search¶

Optional scikit-learn dependency for better retrieval on larger corpora
Authoring agent auto-selects based on estimated corpus size