Files

Mathias Bergqvist a412eee427 docs: add brain MCP migration plan

13 TDD-disciplined tasks moving brain_* and session_log out of the
supervisor pod and into the ingestion pod's MCP handler. Slice 1 of
the larger SKILL.md + routing-MCP architecture migration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-01 09:27:28 +02:00

55 KiB

Raw Blame History

Brain MCP Migration Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Move the brain_* and session_log MCP tools out of the supervisor pod and into the ingestion pod, so Mode 1 (Claude Code, no supervisor) can reach the brain via direct HTTP MCP without the supervisor running.

Architecture: A new ingestion/internal/mcp package adds a notification-aware MCP HTTP handler to the existing ingestion server, alongside its current REST endpoints. The handler dispatches brain_query, brain_write, brain_ingest, brain_ingest_raw, and session_log directly to existing pipeline/search/wiki packages within the same process — no HTTP round-trip to itself, no new module. A NodePort service exposes the MCP endpoint over Tailscale at koala:30330/mcp. The supervisor pod's brain skill remains in place during this slice; deletion is deferred to a later plan.

Tech Stack: Go 1.24, net/http stdlib mux, encoding/json, testify, k3s manifests in the separate infra repo, Flux GitOps for deploy.

File Structure

New files (ingestion module):

ingestion/internal/mcp/server.go — JSON-RPC handler with notification skip
ingestion/internal/mcp/server_test.go — covers parse, dispatch, notifications
ingestion/internal/mcp/handlers.go — brain_* and session_log tool implementations
ingestion/internal/mcp/handlers_test.go — covers each tool against a tmp brain dir
ingestion/internal/session/session.go — copy of internal/session/session.go (will dedupe in supervisor-retirement plan)
ingestion/internal/session/session_test.go — round-trip Append/Read

Modified files (ingestion module):

ingestion/cmd/server/main.go:66-72 — register MCP handler at POST /mcp
ingestion/internal/api/handler.go — extract pure WriteNote helper for reuse from MCP

New files (infra repo):

infra/k3s/apps/supervisor/ingestion-nodeport.yaml — exposes ingestion:3300 as NodePort 30330

Modified files (this repo, root):

.mcp.json — add brain server, leave supervisor until Mode 1 verified
README.md — document the brain MCP endpoint and updated .mcp.json example
CLAUDE.md — note the dual-MCP transitional state

Out of scope for this plan:

Removing the supervisor's brain skill (Plan 7)
Removing internal/session from supervisor module (Plan 7)
Adding brain_search (kb-retrieval not deployed; would be added with that integration)

Task 1: Add MCP server skeleton to ingestion

Files:

Create: ingestion/internal/mcp/server.go
Create: ingestion/internal/mcp/server_test.go

The MCP server in supervisor (internal/mcp/server.go) already has notification handling correct from prior work. We copy that shape — adapted to ingestion's module path — without pulling in supervisor's registry abstraction. Dispatch goes through a single Server.handleCall switch in this slice; if a third MCP server emerges later we can extract a registry then.

Step 1: Write the failing tests

// ingestion/internal/mcp/server_test.go
package mcp_test

import (
	"bytes"
	"encoding/json"
	"net/http"
	"net/http/httptest"
	"strings"
	"testing"

	"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
)

func body(t *testing.T, v any) *bytes.Buffer {
	t.Helper()
	b, err := json.Marshal(v)
	require.NoError(t, err)
	return bytes.NewBuffer(b)
}

func TestServerInitialize(t *testing.T) {
	srv := mcp.NewServer(t.TempDir(), nil, nil)

	req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
		"jsonrpc": "2.0", "id": 1, "method": "initialize",
		"params": map[string]any{},
	}))
	rr := httptest.NewRecorder()
	srv.ServeHTTP(rr, req)

	assert.Equal(t, http.StatusOK, rr.Code)
	var resp map[string]any
	require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
	result := resp["result"].(map[string]any)
	assert.Equal(t, "2024-11-05", result["protocolVersion"])
}

func TestServerToolsList(t *testing.T) {
	srv := mcp.NewServer(t.TempDir(), nil, nil)

	req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
		"jsonrpc": "2.0", "id": 2, "method": "tools/list",
	}))
	rr := httptest.NewRecorder()
	srv.ServeHTTP(rr, req)

	assert.Equal(t, http.StatusOK, rr.Code)
	var resp map[string]any
	require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
	tools := resp["result"].(map[string]any)["tools"].([]any)
	names := make([]string, 0, len(tools))
	for _, t := range tools {
		names = append(names, t.(map[string]any)["name"].(string))
	}
	assert.ElementsMatch(t, []string{
		"brain_query", "brain_write", "brain_ingest_raw", "brain_ingest", "session_log",
	}, names)
}

func TestServerNotificationGetsNoBody(t *testing.T) {
	srv := mcp.NewServer(t.TempDir(), nil, nil)

	req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
		"jsonrpc": "2.0", "method": "notifications/initialized",
	}))
	rr := httptest.NewRecorder()
	srv.ServeHTTP(rr, req)

	assert.Equal(t, http.StatusOK, rr.Code)
	assert.Empty(t, strings.TrimSpace(rr.Body.String()))
}

func TestServerUnknownMethodReturnsError(t *testing.T) {
	srv := mcp.NewServer(t.TempDir(), nil, nil)

	req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
		"jsonrpc": "2.0", "id": 3, "method": "unknown/method",
	}))
	rr := httptest.NewRecorder()
	srv.ServeHTTP(rr, req)

	assert.Equal(t, http.StatusOK, rr.Code)
	var resp map[string]any
	require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
	assert.NotNil(t, resp["error"])
}

Step 2: Run tests to verify they fail

Run: cd ingestion && go test ./internal/mcp/... -v -run 'TestServer' 2>&1 | head -30 Expected: FAIL — package does not exist, mcp.NewServer undefined.

Step 3: Write the minimal server implementation

// ingestion/internal/mcp/server.go
// Package mcp implements an MCP HTTP handler for the ingestion service.
// Exposed tools: brain_query, brain_write, brain_ingest, brain_ingest_raw, session_log.
package mcp

import (
	"context"
	"encoding/json"
	"net/http"

	"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
)

type request struct {
	JSONRPC string          `json:"jsonrpc"`
	ID      any             `json:"id"`
	Method  string          `json:"method"`
	Params  json.RawMessage `json:"params"`
}

type response struct {
	JSONRPC string    `json:"jsonrpc"`
	ID      any       `json:"id,omitempty"`
	Result  any       `json:"result,omitempty"`
	Error   *rpcError `json:"error,omitempty"`
}

type rpcError struct {
	Code    int    `json:"code"`
	Message string `json:"message"`
}

// Server handles MCP JSON-RPC over HTTP for the ingestion service.
type Server struct {
	brainDir string
	pipeline pipeline.Config
	llm      pipeline.CompleteFunc // may be nil in tests
}

// NewServer constructs a Server bound to brainDir. pipelineCfg supplies the
// LLM-backed pipeline; llm may be nil for non-LLM tools only.
func NewServer(brainDir string, pipelineCfg *pipeline.Config, llm pipeline.CompleteFunc) *Server {
	cfg := pipeline.Config{}
	if pipelineCfg != nil {
		cfg = *pipelineCfg
	}
	return &Server{brainDir: brainDir, pipeline: cfg, llm: llm}
}

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	var req request
	if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
		writeError(w, nil, -32700, "parse error")
		return
	}

	// JSON-RPC 2.0 notifications (no id) must not receive a response.
	if req.ID == nil {
		return
	}

	var result any
	var rpcErr *rpcError

	switch req.Method {
	case "initialize":
		result = map[string]any{
			"protocolVersion": "2024-11-05",
			"capabilities":    map[string]any{"tools": map[string]any{}},
			"serverInfo":      map[string]any{"name": "ingestion-brain", "version": "0.1.0"},
		}
	case "tools/list":
		result = map[string]any{"tools": s.tools()}
	case "tools/call":
		var p struct {
			Name      string          `json:"name"`
			Arguments json.RawMessage `json:"arguments"`
		}
		if err := json.Unmarshal(req.Params, &p); err != nil {
			rpcErr = &rpcError{Code: -32602, Message: "invalid params"}
			break
		}
		out, err := s.handleCall(r.Context(), p.Name, p.Arguments)
		if err != nil {
			rpcErr = &rpcError{Code: -32000, Message: err.Error()}
			break
		}
		result = map[string]any{
			"content": []map[string]any{{"type": "text", "text": string(out)}},
		}
	default:
		rpcErr = &rpcError{Code: -32601, Message: "method not found: " + req.Method}
	}

	w.Header().Set("Content-Type", "application/json")
	_ = json.NewEncoder(w).Encode(response{
		JSONRPC: "2.0",
		ID:      req.ID,
		Result:  result,
		Error:   rpcErr,
	})
}

func writeError(w http.ResponseWriter, id any, code int, msg string) {
	w.Header().Set("Content-Type", "application/json")
	_ = json.NewEncoder(w).Encode(response{
		JSONRPC: "2.0",
		ID:      id,
		Error:   &rpcError{Code: code, Message: msg},
	})
}

// handleCall dispatches a tools/call. Stub for Task 1; expanded in later tasks.
func (s *Server) handleCall(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error) {
	return nil, &unknownToolError{name: name}
}

type unknownToolError struct{ name string }

func (e *unknownToolError) Error() string { return "unknown tool: " + e.name }

// ingestion/internal/mcp/handlers.go (placeholder, filled in later tasks)
package mcp

import "encoding/json"

// tools returns the tool descriptors. Schemas are filled per-tool in subsequent tasks.
func (s *Server) tools() []map[string]any {
	str := func(desc string) map[string]any {
		return map[string]any{"type": "string", "description": desc}
	}
	int_ := func(desc string) map[string]any {
		return map[string]any{"type": "integer", "description": desc}
	}
	schema := func(required []string, props map[string]any) json.RawMessage {
		b, _ := json.Marshal(map[string]any{
			"type": "object", "required": required, "properties": props,
		})
		return b
	}

	return []map[string]any{
		{
			"name":        "brain_query",
			"description": "BM25 full-text search across brain/knowledge/ and brain/wiki/ markdown files.",
			"inputSchema": schema([]string{"query"}, map[string]any{
				"query": str("search terms"),
				"limit": int_("max results, default 5"),
			}),
		},
		{
			"name":        "brain_write",
			"description": "Write a raw knowledge note to brain/knowledge/.",
			"inputSchema": schema([]string{"content"}, map[string]any{
				"content":  str("markdown content"),
				"filename": str("optional filename"),
				"type":     str("optional frontmatter type"),
				"domain":   str("optional frontmatter domain"),
			}),
		},
		{
			"name":        "brain_ingest_raw",
			"description": "Ingest pre-structured pages into the brain wiki, bypassing the LLM extraction step.",
			"inputSchema": schema([]string{"source", "pages"}, map[string]any{
				"source":  str("source name"),
				"pages":   map[string]any{"type": "array"},
				"dry_run": map[string]any{"type": "boolean"},
			}),
		},
		{
			"name":        "brain_ingest",
			"description": "Ingest content into the brain wiki via the LLM extraction pipeline.",
			"inputSchema": schema([]string{}, map[string]any{
				"content": str("raw content; required when path is empty"),
				"source":  str("source name; required when path is empty"),
				"path":    str("file path; mutually exclusive with content+source"),
				"dry_run": map[string]any{"type": "boolean"},
			}),
		},
		{
			"name":        "session_log",
			"description": "Append a structured entry to brain/sessions/<session_id>.jsonl.",
			"inputSchema": schema([]string{"session_id"}, map[string]any{
				"session_id":   str("session identifier"),
				"skill":        str("skill name"),
				"phase":        str("phase within the skill"),
				"project_root": str("absolute project root"),
				"final_status": str("ok | error | skipped"),
				"file_path":    str("optional file produced"),
				"model_used":   str("optional model identifier"),
				"duration_ms":  int_("optional duration in ms"),
				"message":      str("optional free-text"),
			}),
		},
	}
}

Step 4: Run tests to verify they pass

Run: cd ingestion && go test ./internal/mcp/... -v -run 'TestServer' 2>&1 | tail -20 Expected: All four TestServer* tests PASS.

Step 5: Commit

cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
git add ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): add MCP server skeleton with tools/list

Adds an MCP HTTP handler under ingestion/internal/mcp. Implements
initialize, tools/list, and the JSON-RPC notification skip from prior
work. Tool dispatch is stubbed (returns unknown-tool error) and will be
filled in by subsequent tasks.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 2: Add session package to ingestion

Files:

Create: ingestion/internal/session/session.go
Create: ingestion/internal/session/session_test.go

We deliberately copy internal/session/session.go from supervisor rather than introduce a third Go module. Plan 7 (supervisor retirement) will delete the supervisor copy. Until then both exist; they will not drift because no edits happen during transition.

Step 1: Write the failing test

// ingestion/internal/session/session_test.go
package session_test

import (
	"path/filepath"
	"testing"
	"time"

	"github.com/mathiasbq/hyperguild/ingestion/internal/session"
	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
)

func TestAppendAndRead(t *testing.T) {
	dir := t.TempDir()
	sid := "test-session"

	e1 := session.Entry{
		SessionID:   sid,
		Timestamp:   time.Now().UTC().Truncate(time.Second),
		Skill:       "tdd",
		Phase:       "red",
		FinalStatus: "ok",
	}
	e2 := session.Entry{
		SessionID:   sid,
		Timestamp:   time.Now().UTC().Truncate(time.Second),
		Skill:       "tdd",
		Phase:       "green",
		FinalStatus: "ok",
	}

	require.NoError(t, session.Append(dir, sid, e1))
	require.NoError(t, session.Append(dir, sid, e2))

	got, err := session.Read(dir, sid)
	require.NoError(t, err)
	require.Len(t, got, 2)
	assert.Equal(t, "red", got[0].Phase)
	assert.Equal(t, "green", got[1].Phase)

	// File path is sessionsDir/<sid>.jsonl
	_, statErr := filepath.Abs(filepath.Join(dir, sid+".jsonl"))
	require.NoError(t, statErr)
}

func TestReadMissingReturnsEmpty(t *testing.T) {
	got, err := session.Read(t.TempDir(), "nope")
	require.NoError(t, err)
	assert.Empty(t, got)
}

Step 2: Run test to verify it fails

Run: cd ingestion && go test ./internal/session/... -v 2>&1 | head -20 Expected: FAIL — package does not exist.

Step 3: Write the implementation

Copy /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration/internal/session/session.go verbatim into ingestion/internal/session/session.go, changing only the package comment header to reference the ingestion path.

// ingestion/internal/session/session.go
package session

import (
	"bufio"
	"encoding/json"
	"errors"
	"fmt"
	"io/fs"
	"os"
	"path/filepath"
	"time"
)

// Entry is one skill invocation record, appended to the session JSONL log.
type Entry struct {
	SessionID   string          `json:"session_id"`
	Timestamp   time.Time       `json:"timestamp"`
	Skill       string          `json:"skill"`
	Phase       string          `json:"phase,omitempty"`
	ProjectRoot string          `json:"project_root,omitempty"`
	Input       json.RawMessage `json:"input,omitempty"`
	Attempts    []Attempt       `json:"attempts,omitempty"`
	FinalStatus string          `json:"final_status"`
	FilePath    string          `json:"file_path,omitempty"`
	ModelUsed   string          `json:"model_used,omitempty"`
	DurationMs  int64           `json:"duration_ms,omitempty"`
	Message     string          `json:"message,omitempty"`
}

// Attempt represents one subprocess invocation within a skill call.
type Attempt struct {
	Attempt       int    `json:"attempt"`
	Model         string `json:"model"`
	Tier          string `json:"tier"`
	DurationMs    int64  `json:"duration_ms"`
	WarmStart     bool   `json:"warm_start"`
	Verified      bool   `json:"verified"`
	Verdict       string `json:"verdict,omitempty"`
	Feedback      string `json:"feedback,omitempty"`
	OutputSummary string `json:"output_summary,omitempty"`
	RunnerOutput  string `json:"runner_output,omitempty"`
}

// Append writes entry as a single JSON line to sessionsDir/{sessionID}.jsonl.
func Append(sessionsDir, sessionID string, entry Entry) error {
	if err := os.MkdirAll(sessionsDir, 0o755); err != nil {
		return fmt.Errorf("create sessions dir: %w", err)
	}
	path := filepath.Join(sessionsDir, sessionID+".jsonl")
	f, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
	if err != nil {
		return fmt.Errorf("open session log: %w", err)
	}
	line, err := json.Marshal(entry)
	if err != nil {
		_ = f.Close()
		return fmt.Errorf("marshal entry: %w", err)
	}
	if _, err = fmt.Fprintf(f, "%s\n", line); err != nil {
		_ = f.Close()
		return fmt.Errorf("write entry: %w", err)
	}
	if err = f.Close(); err != nil {
		return fmt.Errorf("close session log: %w", err)
	}
	return nil
}

// Read returns all entries for sessionID. Returns empty slice if no log exists.
func Read(sessionsDir, sessionID string) ([]Entry, error) {
	path := filepath.Join(sessionsDir, sessionID+".jsonl")
	f, err := os.Open(path)
	if errors.Is(err, fs.ErrNotExist) {
		return []Entry{}, nil
	}
	if err != nil {
		return nil, fmt.Errorf("open session log: %w", err)
	}
	defer f.Close() //nolint:errcheck

	var entries []Entry
	scanner := bufio.NewScanner(f)
	scanner.Buffer(make([]byte, 0, 256*1024), 1<<20)
	for scanner.Scan() {
		line := scanner.Bytes()
		if len(line) == 0 {
			continue
		}
		var e Entry
		if err := json.Unmarshal(line, &e); err != nil {
			return nil, fmt.Errorf("parse entry: %w", err)
		}
		entries = append(entries, e)
	}
	return entries, scanner.Err()
}

Step 4: Run test to verify it passes

Run: cd ingestion && go test ./internal/session/... -v 2>&1 | tail -10 Expected: Both tests PASS.

Step 5: Commit

git add ingestion/internal/session
git commit -m "$(cat <<'EOF'
feat(ingestion): add session package for JSONL log persistence

Copy of internal/session from the supervisor module — the ingestion
service needs it for the upcoming session_log MCP tool. The supervisor
copy will be removed in the supervisor-retirement plan; until then
the two packages are intentionally identical and pinned (no edits).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 3: Implement brain_query MCP tool

Files:

Modify: ingestion/internal/mcp/handlers.go (add brainQuery method + dispatch)
Modify: ingestion/internal/mcp/server.go (wire handleCall to call brainQuery)
Create test in: ingestion/internal/mcp/handlers_test.go

brain_query calls the existing pure function search.Query(brainDir, query, limit). No HTTP self-call needed.

Step 1: Write the failing test

// ingestion/internal/mcp/handlers_test.go
package mcp_test

import (
	"bytes"
	"encoding/json"
	"net/http"
	"net/http/httptest"
	"os"
	"path/filepath"
	"testing"

	"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
)

func toolCall(t *testing.T, srv http.Handler, name string, args map[string]any) map[string]any {
	t.Helper()
	bodyBytes, err := json.Marshal(map[string]any{
		"jsonrpc": "2.0", "id": 1, "method": "tools/call",
		"params": map[string]any{"name": name, "arguments": args},
	})
	require.NoError(t, err)
	req := httptest.NewRequest(http.MethodPost, "/mcp", bytes.NewReader(bodyBytes))
	rr := httptest.NewRecorder()
	srv.ServeHTTP(rr, req)
	require.Equal(t, http.StatusOK, rr.Code)
	var resp map[string]any
	require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
	return resp
}

func TestBrainQueryReturnsResults(t *testing.T) {
	brainDir := t.TempDir()
	knowledge := filepath.Join(brainDir, "knowledge")
	require.NoError(t, os.MkdirAll(knowledge, 0o755))
	require.NoError(t, os.WriteFile(
		filepath.Join(knowledge, "tdd.md"),
		[]byte("# TDD\n\nTest-driven development is iterative.\n"),
		0o644,
	))

	srv := mcp.NewServer(brainDir, nil, nil)
	resp := toolCall(t, srv, "brain_query", map[string]any{"query": "tdd"})

	require.Nil(t, resp["error"])
	result := resp["result"].(map[string]any)
	content := result["content"].([]any)
	require.NotEmpty(t, content)
	text := content[0].(map[string]any)["text"].(string)
	assert.Contains(t, text, "tdd.md")
}

Step 2: Run test to verify it fails

Run: cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainQuery' 2>&1 | head -20 Expected: FAIL — unknown tool: brain_query.

Step 3: Implement brainQuery

Add to ingestion/internal/mcp/handlers.go:

// (append to existing handlers.go)

import (
	"context"
	"encoding/json"
	"fmt"

	"github.com/mathiasbq/hyperguild/ingestion/internal/search"
)

type brainQueryArgs struct {
	Query string `json:"query"`
	Limit int    `json:"limit,omitempty"`
}

func (s *Server) brainQuery(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
	var a brainQueryArgs
	if err := json.Unmarshal(args, &a); err != nil {
		return nil, fmt.Errorf("parse args: %w", err)
	}
	if a.Query == "" {
		return nil, fmt.Errorf("query is required")
	}
	if a.Limit == 0 {
		a.Limit = 5
	}
	results, err := search.Query(s.brainDir, a.Query, a.Limit)
	if err != nil {
		return nil, fmt.Errorf("search: %w", err)
	}
	return json.Marshal(map[string]any{"results": results})
}

Update ingestion/internal/mcp/server.go's handleCall to dispatch:

// ingestion/internal/mcp/server.go — replace handleCall and unknownToolError block

func (s *Server) handleCall(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error) {
	switch name {
	case "brain_query":
		return s.brainQuery(ctx, args)
	default:
		return nil, fmt.Errorf("unknown tool: %s", name)
	}
}

Add the "fmt" import to server.go and remove the now-unused unknownToolError type and its Error() method.

Step 4: Run test to verify it passes

Run: cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainQuery' 2>&1 | tail -10 Expected: PASS.

Step 5: Commit

git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/handlers_test.go ingestion/internal/mcp/server.go
git commit -m "$(cat <<'EOF'
feat(ingestion): implement brain_query MCP tool

Wraps the existing search.Query function. Same BM25 over
brain/knowledge/ and brain/wiki/ that the HTTP /query endpoint serves.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 4: Extract WriteNote helper and implement brain_write

Files:

Modify: ingestion/internal/api/handler.go (extract WriteNote from Handler.Write)
Modify: ingestion/internal/mcp/handlers.go (call api.WriteNote)
Modify: ingestion/internal/mcp/server.go (dispatch brain_write)
Modify: ingestion/internal/api/handler_test.go (regression: existing /write still works)
Modify: ingestion/internal/mcp/handlers_test.go (new: brain_write writes a file)

The current Handler.Write mixes JSON decoding, validation, frontmatter construction, path safety, and disk writing. We extract a pure WriteNote helper so MCP and HTTP both call it.

Step 1: Write the failing tests

Add to ingestion/internal/mcp/handlers_test.go:

func TestBrainWriteCreatesFile(t *testing.T) {
	brainDir := t.TempDir()
	srv := mcp.NewServer(brainDir, nil, nil)

	resp := toolCall(t, srv, "brain_write", map[string]any{
		"content":  "# Test\n\nbody",
		"filename": "test.md",
		"type":     "note",
		"domain":   "personal",
	})
	require.Nil(t, resp["error"])

	// File should exist under brain/knowledge/test.md with frontmatter
	got, err := os.ReadFile(filepath.Join(brainDir, "knowledge", "test.md"))
	require.NoError(t, err)
	assert.Contains(t, string(got), "type: note")
	assert.Contains(t, string(got), "domain: personal")
	assert.Contains(t, string(got), "# Test")
}

func TestBrainWriteRejectsTraversal(t *testing.T) {
	brainDir := t.TempDir()
	srv := mcp.NewServer(brainDir, nil, nil)

	resp := toolCall(t, srv, "brain_write", map[string]any{
		"content":  "x",
		"filename": "../escape.md",
	})
	assert.NotNil(t, resp["error"])
}

Step 2: Run tests to verify they fail

Run: cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainWrite' 2>&1 | head -20 Expected: FAIL — unknown tool: brain_write.

Step 3: Extract WriteNote in api/handler.go

Replace the body of Handler.Write with a thin wrapper that calls a new exported function WriteNote. Read ingestion/internal/api/handler.go:88-142 first to confirm the exact code.

// ingestion/internal/api/handler.go — add new exported function before func (h *Handler) Write
// and replace the body of Write to call WriteNote.

// WriteNote writes a markdown file to brainDir/knowledge/<filename>, optionally
// prefixed with YAML frontmatter built from typ and domain. Returns the path
// relative to brainDir (forward-slashed). Filename traversal is rejected.
func WriteNote(brainDir, content, filename, typ, domain string) (string, error) {
	if content == "" {
		return "", fmt.Errorf("content is required")
	}
	if filename == "" {
		filename = fmt.Sprintf("%s-auto.md", time.Now().UTC().Format("2006-01-02-150405"))
	}

	rawDir := filepath.Join(brainDir, "knowledge")
	if err := os.MkdirAll(rawDir, 0o755); err != nil {
		return "", fmt.Errorf("create raw dir: %w", err)
	}

	finalContent := content
	if typ != "" || domain != "" {
		var fm strings.Builder
		fm.WriteString("---\n")
		if typ != "" {
			fmt.Fprintf(&fm, "type: %s\n", typ)
		}
		if domain != "" {
			fmt.Fprintf(&fm, "domain: %s\n", domain)
		}
		fm.WriteString("---\n")
		finalContent = fm.String() + content
	}

	base := filepath.Base(filename)
	if !strings.HasSuffix(base, ".md") {
		base += ".md"
	}
	dest := filepath.Join(rawDir, base)
	if !strings.HasPrefix(filepath.Clean(dest)+string(os.PathSeparator),
		filepath.Clean(rawDir)+string(os.PathSeparator)) {
		return "", fmt.Errorf("invalid filename")
	}
	if err := os.WriteFile(dest, []byte(finalContent), 0o644); err != nil {
		return "", fmt.Errorf("write: %w", err)
	}

	rel, _ := filepath.Rel(brainDir, dest)
	return filepath.ToSlash(rel), nil
}

Replace Handler.Write body to call WriteNote:

func (h *Handler) Write(w http.ResponseWriter, r *http.Request) {
	var req writeRequest
	if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
		writeError(w, http.StatusBadRequest, "invalid JSON")
		return
	}
	relPath, err := WriteNote(h.brainDir, req.Content, req.Filename, req.Type, req.Domain)
	if err != nil {
		h.logger.Error("write failed", "err", err)
		writeError(w, http.StatusBadRequest, err.Error())
		return
	}
	writeJSON(w, map[string]string{"path": relPath})
}

Step 4: Add brainWrite handler and dispatch

Append to ingestion/internal/mcp/handlers.go:

import (
	// add to existing import block
	"github.com/mathiasbq/hyperguild/ingestion/internal/api"
)

type brainWriteArgs struct {
	Content  string `json:"content"`
	Filename string `json:"filename,omitempty"`
	Type     string `json:"type,omitempty"`
	Domain   string `json:"domain,omitempty"`
}

func (s *Server) brainWrite(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
	var a brainWriteArgs
	if err := json.Unmarshal(args, &a); err != nil {
		return nil, fmt.Errorf("parse args: %w", err)
	}
	relPath, err := api.WriteNote(s.brainDir, a.Content, a.Filename, a.Type, a.Domain)
	if err != nil {
		return nil, err
	}
	return json.Marshal(map[string]string{"path": relPath})
}

Update handleCall switch:

case "brain_write":
	return s.brainWrite(ctx, args)

Step 5: Run all ingestion tests

Run: cd ingestion && go test -count=1 ./... 2>&1 | tail -15 Expected: All packages PASS, including the existing api tests (regression: HTTP /write still works) and the new mcp tests.

Step 6: Commit

git add ingestion/internal/api/handler.go ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): extract WriteNote helper and add brain_write MCP tool

api.WriteNote captures the file-write logic that was previously inline
in Handler.Write. The existing HTTP endpoint now delegates to it; the
new MCP brain_write tool reuses the same function. Path-traversal
guard is preserved.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 5: Implement brain_ingest_raw

Files:

Modify: ingestion/internal/mcp/handlers.go (add brainIngestRaw)
Modify: ingestion/internal/mcp/server.go (dispatch)
Modify: ingestion/internal/mcp/handlers_test.go (smoke test against tmp brain)

pipeline.RunRaw(brainDir, source, pages, dryRun) is already pure — direct call.

Step 1: Write the failing test

Add to ingestion/internal/mcp/handlers_test.go:

import (
	// existing + add:
	"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
)

func TestBrainIngestRawDryRun(t *testing.T) {
	brainDir := t.TempDir()
	require.NoError(t, os.MkdirAll(filepath.Join(brainDir, "wiki", "concepts"), 0o755))
	srv := mcp.NewServer(brainDir, nil, nil)

	resp := toolCall(t, srv, "brain_ingest_raw", map[string]any{
		"source":  "test-source",
		"dry_run": true,
		"pages": []map[string]any{
			{
				"title":   "Test Concept",
				"type":    "concept",
				"content": "## Definition\nA test concept.",
			},
		},
	})
	require.Nil(t, resp["error"])
	result := resp["result"].(map[string]any)
	content := result["content"].([]any)
	text := content[0].(map[string]any)["text"].(string)

	var parsed struct {
		Pages []string `json:"pages"`
	}
	require.NoError(t, json.Unmarshal([]byte(text), &parsed))
	assert.Contains(t, parsed.Pages[0], "wiki/concepts/test-concept.md")

	// dry_run: no file should exist
	_, err := os.Stat(filepath.Join(brainDir, "wiki", "concepts", "test-concept.md"))
	assert.True(t, os.IsNotExist(err))
}

var _ = pipeline.RawPage{} // keep import linked for future tests

Step 2: Run test, verify failure

Run: cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainIngestRaw' 2>&1 | head -20 Expected: FAIL — unknown tool: brain_ingest_raw.

Step 3: Implement

Append to ingestion/internal/mcp/handlers.go:

type brainIngestRawArgs struct {
	Source string             `json:"source"`
	Pages  []pipeline.RawPage `json:"pages"`
	DryRun bool               `json:"dry_run,omitempty"`
}

func (s *Server) brainIngestRaw(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
	var a brainIngestRawArgs
	if err := json.Unmarshal(args, &a); err != nil {
		return nil, fmt.Errorf("parse args: %w", err)
	}
	if a.Source == "" {
		return nil, fmt.Errorf("source is required")
	}
	if len(a.Pages) == 0 {
		return nil, fmt.Errorf("pages must be non-empty")
	}
	result, err := pipeline.RunRaw(s.brainDir, a.Source, a.Pages, a.DryRun)
	if err != nil {
		return nil, fmt.Errorf("ingest: %w", err)
	}
	pages := result.Pages
	if pages == nil {
		pages = []string{}
	}
	warnings := result.Warnings
	if warnings == nil {
		warnings = []string{}
	}
	return json.Marshal(map[string]any{"pages": pages, "warnings": warnings})
}

Update import block to include pipeline package.

Update handleCall switch:

case "brain_ingest_raw":
	return s.brainIngestRaw(ctx, args)

Step 4: Run tests

Run: cd ingestion && go test -count=1 ./internal/mcp/... 2>&1 | tail -10 Expected: PASS.

Step 5: Commit

git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): implement brain_ingest_raw MCP tool

Wraps pipeline.RunRaw directly. Same dry-run semantics as the HTTP
/ingest-raw endpoint.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 6: Implement brain_ingest (LLM path)

Files:

Modify: ingestion/internal/mcp/handlers.go (add brainIngest)
Modify: ingestion/internal/mcp/server.go (dispatch + LLM availability check)
Modify: ingestion/internal/mcp/handlers_test.go (path-vs-content+source mutex test)

brain_ingest takes either path (file) or content+source. Calls pipeline.Run which uses the LLM. The MCP server's s.pipeline config holds the Complete function.

Step 1: Write the failing tests

Add to ingestion/internal/mcp/handlers_test.go:

func TestBrainIngestRejectsBoth(t *testing.T) {
	brainDir := t.TempDir()
	srv := mcp.NewServer(brainDir, nil, nil)

	resp := toolCall(t, srv, "brain_ingest", map[string]any{
		"content": "x",
		"source":  "y",
		"path":    "/tmp/foo.md",
	})
	assert.NotNil(t, resp["error"])
}

func TestBrainIngestRequiresOne(t *testing.T) {
	brainDir := t.TempDir()
	srv := mcp.NewServer(brainDir, nil, nil)

	resp := toolCall(t, srv, "brain_ingest", map[string]any{})
	assert.NotNil(t, resp["error"])
}

Step 2: Run, verify FAIL

Run: cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainIngest' 2>&1 | head -20 Expected: FAIL — unknown tool: brain_ingest.

Step 3: Implement

Append to ingestion/internal/mcp/handlers.go:

import (
	// add:
	"github.com/mathiasbq/hyperguild/ingestion/internal/extract"
	"path/filepath"
	"strings"
)

type brainIngestArgs struct {
	Content string `json:"content,omitempty"`
	Source  string `json:"source,omitempty"`
	Path    string `json:"path,omitempty"`
	DryRun  bool   `json:"dry_run,omitempty"`
}

func (s *Server) brainIngest(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
	var a brainIngestArgs
	if err := json.Unmarshal(args, &a); err != nil {
		return nil, fmt.Errorf("parse args: %w", err)
	}
	if a.Path != "" && a.Content != "" {
		return nil, fmt.Errorf("path and content+source are mutually exclusive")
	}
	if a.Path == "" && a.Content == "" {
		return nil, fmt.Errorf("either path or content+source is required")
	}
	if s.pipeline.Complete == nil {
		return nil, fmt.Errorf("LLM not configured: set INGEST_LLM_URL")
	}

	if a.Path != "" {
		text, err := extract.Text(a.Path)
		if err != nil {
			return nil, fmt.Errorf("extract: %w", err)
		}
		source := a.Source
		if source == "" {
			source = filepath.Base(strings.TrimSuffix(a.Path, filepath.Ext(a.Path)))
		}
		return s.runIngest(ctx, text, source, a.DryRun)
	}
	if a.Source == "" {
		return nil, fmt.Errorf("source is required when content is provided")
	}
	return s.runIngest(ctx, a.Content, a.Source, a.DryRun)
}

func (s *Server) runIngest(ctx context.Context, content, source string, dryRun bool) (json.RawMessage, error) {
	result, err := pipeline.Run(ctx, s.pipeline, s.brainDir, content, source, dryRun)
	if err != nil {
		return nil, fmt.Errorf("ingest: %w", err)
	}
	pages := result.Pages
	if pages == nil {
		pages = []string{}
	}
	warnings := result.Warnings
	if warnings == nil {
		warnings = []string{}
	}
	return json.Marshal(map[string]any{"pages": pages, "warnings": warnings})
}

Update handleCall switch:

case "brain_ingest":
	return s.brainIngest(ctx, args)

Step 4: Run tests

Run: cd ingestion && go test -count=1 ./internal/mcp/... 2>&1 | tail -10 Expected: All PASS.

Step 5: Commit

git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): implement brain_ingest MCP tool

Wraps pipeline.Run with the existing LLM client. Mirrors the HTTP
/ingest and /ingest-path semantics — accepts either path or
content+source, validates mutual exclusion, surfaces an explicit error
when the LLM client is not configured (test-mode).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 7: Implement session_log

Files:

Modify: ingestion/internal/mcp/handlers.go (add sessionLog)
Modify: ingestion/internal/mcp/server.go (dispatch)
Modify: ingestion/internal/mcp/handlers_test.go (verify JSONL line written)

Writes to ${BRAIN_DIR}/sessions/<session_id>.jsonl via the new ingestion/internal/session package.

Step 1: Write the failing test

Add to ingestion/internal/mcp/handlers_test.go:

func TestSessionLogAppends(t *testing.T) {
	brainDir := t.TempDir()
	srv := mcp.NewServer(brainDir, nil, nil)

	resp := toolCall(t, srv, "session_log", map[string]any{
		"session_id":   "session-x",
		"skill":        "tdd",
		"phase":        "red",
		"final_status": "ok",
	})
	require.Nil(t, resp["error"])

	got, err := os.ReadFile(filepath.Join(brainDir, "sessions", "session-x.jsonl"))
	require.NoError(t, err)
	assert.Contains(t, string(got), `"skill":"tdd"`)
	assert.Contains(t, string(got), `"phase":"red"`)
}

func TestSessionLogRequiresSessionID(t *testing.T) {
	srv := mcp.NewServer(t.TempDir(), nil, nil)
	resp := toolCall(t, srv, "session_log", map[string]any{"skill": "tdd"})
	assert.NotNil(t, resp["error"])
}

Step 2: Run, verify FAIL

Run: cd ingestion && go test ./internal/mcp/... -v -run 'TestSessionLog' 2>&1 | head -20 Expected: FAIL.

Step 3: Implement

Append to ingestion/internal/mcp/handlers.go:

import (
	// add:
	"path/filepath"
	"time"
	"github.com/mathiasbq/hyperguild/ingestion/internal/session"
)

type sessionLogArgs struct {
	SessionID   string `json:"session_id"`
	Skill       string `json:"skill,omitempty"`
	Phase       string `json:"phase,omitempty"`
	ProjectRoot string `json:"project_root,omitempty"`
	FinalStatus string `json:"final_status,omitempty"`
	FilePath    string `json:"file_path,omitempty"`
	ModelUsed   string `json:"model_used,omitempty"`
	DurationMs  int64  `json:"duration_ms,omitempty"`
	Message     string `json:"message,omitempty"`
}

func (s *Server) sessionLog(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
	var a sessionLogArgs
	if err := json.Unmarshal(args, &a); err != nil {
		return nil, fmt.Errorf("parse args: %w", err)
	}
	if a.SessionID == "" {
		return nil, fmt.Errorf("session_id is required")
	}
	entry := session.Entry{
		SessionID:   a.SessionID,
		Timestamp:   time.Now().UTC(),
		Skill:       a.Skill,
		Phase:       a.Phase,
		ProjectRoot: a.ProjectRoot,
		FinalStatus: a.FinalStatus,
		FilePath:    a.FilePath,
		ModelUsed:   a.ModelUsed,
		DurationMs:  a.DurationMs,
		Message:     a.Message,
	}
	dir := filepath.Join(s.brainDir, "sessions")
	if err := session.Append(dir, a.SessionID, entry); err != nil {
		return nil, fmt.Errorf("append: %w", err)
	}
	return json.Marshal(map[string]string{"status": "ok", "session_id": a.SessionID})
}

Update handleCall:

case "session_log":
	return s.sessionLog(ctx, args)

Step 4: Run tests

Run: cd ingestion && go test -count=1 ./internal/mcp/... 2>&1 | tail -10 Expected: PASS.

Step 5: Commit

git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): implement session_log MCP tool

Appends a JSON line to brainDir/sessions/<session_id>.jsonl using the
copied session package. Required for upcoming pass-rate logging.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 8: Wire MCP handler into ingestion server

Files:

Modify: ingestion/cmd/server/main.go:66-72 (mount MCP at POST /mcp)
Step 1: Write a smoke test that exercises the live handler over HTTP

Add a new file ingestion/internal/mcp/integration_test.go:

package mcp_test

import (
	"bytes"
	"encoding/json"
	"io"
	"net/http"
	"net/http/httptest"
	"testing"

	"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
)

func TestMCPMountedHandler(t *testing.T) {
	srv := mcp.NewServer(t.TempDir(), nil, nil)
	mux := http.NewServeMux()
	mux.Handle("POST /mcp", srv)

	ts := httptest.NewServer(mux)
	defer ts.Close()

	body, _ := json.Marshal(map[string]any{
		"jsonrpc": "2.0", "id": 1, "method": "tools/list",
	})
	resp, err := http.Post(ts.URL+"/mcp", "application/json", bytes.NewReader(body))
	require.NoError(t, err)
	defer resp.Body.Close()
	assert.Equal(t, http.StatusOK, resp.StatusCode)

	out, _ := io.ReadAll(resp.Body)
	assert.Contains(t, string(out), `"brain_query"`)
}

Step 2: Run, expect FAIL until step 3

This test passes today (it constructs its own mux). Skip step 2 — the test verifies the wiring shape; the actual wiring change in step 3 ensures cmd/server/main.go also mounts it.

Run: cd ingestion && go test ./internal/mcp/... -v -run 'TestMCPMountedHandler' 2>&1 | tail -10 Expected: PASS (test is module-local and constructs its own mux).

Step 3: Wire into main.go

Modify ingestion/cmd/server/main.go:

Add to imports:

"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"

After h := api.NewHandler(...) and before the mux := http.NewServeMux() block, construct the MCP server:

mcpSrv := mcp.NewServer(brainDir, &pipelineCfg, llmClient.Complete)

Add the route after the existing POST handlers (around line 72):

mux.Handle("POST /mcp", mcpSrv)

Update the startup log line to mention MCP:

logger.Info("ingestion server starting",
	"addr", addr,
	"brain_dir", brainDir,
	"llm_url", llmURL,
	"llm_model", llmModel,
	"chunk_size", chunkSize,
	"watch_interval", watchIntervalLog,
	"mcp_enabled", true,
)

Step 4: Build and run locally as a smoke test

cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
mkdir -p /tmp/brain-mcp-test/{wiki,sessions,knowledge}
INGEST_BRAIN_DIR=/tmp/brain-mcp-test \
INGEST_PORT=33301 \
INGEST_WATCH_INTERVAL=0 \
go run ./ingestion/cmd/server/ &
SERVER_PID=$!
sleep 2

# tools/list
curl -sS -X POST http://localhost:33301/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | head -c 500
echo ""

# notifications/initialized → empty body
curl -sS -i -X POST http://localhost:33301/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","method":"notifications/initialized"}' | head -10
echo ""

# brain_query smoke
curl -sS -X POST http://localhost:33301/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"brain_query","arguments":{"query":"x"}}}' \
  | head -c 200
echo ""

kill $SERVER_PID
wait $SERVER_PID 2>/dev/null
rm -rf /tmp/brain-mcp-test

Expected:

tools/list returns JSON with all 5 brain tool names.
notifications/initialized returns HTTP/1.1 200 with Content-Length: 0.
brain_query returns {"results":null} (empty brain).
Step 5: Run full ingestion test suite

Run: cd ingestion && go test -race -count=1 ./... 2>&1 | tail -15 Expected: All packages PASS.

Step 6: Commit

git add ingestion/cmd/server/main.go ingestion/internal/mcp/integration_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): mount MCP handler at POST /mcp

The ingestion server now exposes both REST and MCP on the same port
(3300). MCP shares brainDir, pipeline config, and LLM client with the
REST handlers — single source of process state.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 9: Add NodePort service in infra repo

Files (in ~/Documents/local-dev/AI/infra):

Create: infra/k3s/apps/supervisor/ingestion-nodeport.yaml
Modify: infra/k3s/apps/supervisor/kustomization.yaml

This is in a different repo. The branch and PR happen there. Confirm with the user before pushing — infra repo changes touch live cluster routing.

Step 1: Create the NodePort manifest

Read infra/k3s/apps/supervisor/supervisor-nodeport.yaml first to mirror its style.

# infra/k3s/apps/supervisor/ingestion-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
  name: ingestion-nodeport
  namespace: supervisor
spec:
  type: NodePort
  selector:
    app: ingestion
  ports:
    - name: http
      port: 3300
      targetPort: 3300
      nodePort: 30330

Step 2: Add to kustomization.yaml

Modify infra/k3s/apps/supervisor/kustomization.yaml resources list:

resources:
  - namespace.yaml
  - deployment.yaml
  - service.yaml
  - secrets.enc.yaml
  - supervisor-nodeport.yaml
  - ingestion-deployment.yaml
  - ingestion-service.yaml
  - ingestion-nodeport.yaml   # added

Step 3: Commit (in infra repo)

cd /Users/mathias/Documents/local-dev/AI/infra
git checkout -b feat/ingestion-nodeport
git add k3s/apps/supervisor/ingestion-nodeport.yaml k3s/apps/supervisor/kustomization.yaml
git commit -m "$(cat <<'EOF'
feat(supervisor): expose ingestion as NodePort 30330 for direct MCP

Pairs with hyperguild's brain MCP migration — Claude Code can now
reach brain MCP at http://koala:30330/mcp without going through the
supervisor pod.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Step 4: Confirm with user before push

Pause and ask: "Push the infra change to gitea origin? Flux will reconcile within ~30s and add the NodePort service. Confirm?"

If confirmed:

cd /Users/mathias/Documents/local-dev/AI/infra
git push origin feat/ingestion-nodeport

(Or merge to main per your existing GitOps workflow — depends on whether feature branches are reconciled by Flux. Existing pattern is direct-to-main commits.)

Step 5: Verify NodePort live

kubectl --request-timeout=10s -n supervisor get svc ingestion-nodeport 2>&1

Expected output includes a service with TYPE=NodePort and PORT(S) 3300:30330/TCP.

Task 10: End-to-end smoke test against deployed brain MCP

Prerequisite: Hyperguild commits from Tasks 1-8 are pushed to gitea origin and the new ingestion image has rolled out to the cluster (~2 minutes after push, per CD pattern).

Step 1: Push the hyperguild changes

cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
git push origin feat/brain-mcp-migration

Step 2: Open a PR to main, get it reviewed, merge

(Manual step. Or fast-forward to main if reviewing solo.)

Step 3: Wait for image rollout

TARGET=$(git -C /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration rev-parse HEAD)
for i in $(seq 1 30); do
  CURRENT=$(kubectl --request-timeout=5s -n supervisor get deploy ingestion -o jsonpath='{.spec.template.spec.containers[0].image}' 2>/dev/null | sed 's/.*://')
  if [ "$CURRENT" = "$TARGET" ]; then
    echo "[$i] image rolled to $CURRENT"
    break
  fi
  echo "[$i] still on ${CURRENT:0:12} — waiting 10s"
  sleep 10
done
kubectl --request-timeout=8s -n supervisor get pods -l app=ingestion -o wide

Step 4: Smoke test the live endpoint

echo "=== initialize ==="
curl -sS -X POST http://koala:30330/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"smoke","version":"0"}}}'
echo ""

echo "=== tools/list ==="
curl -sS -X POST http://koala:30330/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | head -c 600
echo ""

echo "=== brain_query (existing brain) ==="
curl -sS -X POST http://koala:30330/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"brain_query","arguments":{"query":"tdd","limit":2}}}'
echo ""

echo "=== session_log ==="
curl -sS -X POST http://koala:30330/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"session_log","arguments":{"session_id":"smoke-test","skill":"plan-execution","phase":"verify","final_status":"ok"}}}'
echo ""

Expected:

initialize returns protocolVersion: "2024-11-05" and serverInfo.name: "ingestion-brain".
tools/list returns 5 tools.
brain_query returns whatever's in the brain (or {"results":null} if empty).
session_log returns {"status":"ok","session_id":"smoke-test"}.

If any of these fail: do not proceed to Task 11. Investigate the failure first.

Task 11: Update .mcp.json to use brain MCP

Files:

Modify: .mcp.json (in repo root)

The supervisor MCP entry stays for now (it still works, and the migration of skill workers is a future plan). We add brain as a second MCP server.

Step 1: Read current .mcp.json

cat /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration/.mcp.json

Expected:

{
  "mcpServers": {
    "supervisor": {
      "type": "http",
      "url": "http://koala:30320/mcp"
    }
  }
}

Step 2: Add brain alongside

{
  "mcpServers": {
    "supervisor": {
      "type": "http",
      "url": "http://koala:30320/mcp"
    },
    "brain": {
      "type": "http",
      "url": "http://koala:30330/mcp"
    }
  }
}

Step 3: User restarts Claude Code in this project

Manual handoff to user: "Restart Claude Code. Then /mcp should list two servers: supervisor and brain. The brain server should expose 5 tools (brain_query, brain_write, brain_ingest, brain_ingest_raw, session_log). Try a brain_query and confirm it returns results."

Step 4: Commit only after user confirms

git add .mcp.json
git commit -m "$(cat <<'EOF'
chore: add brain MCP server alongside supervisor

The brain MCP at koala:30330 hosts the brain_* and session_log tools
formerly on supervisor. Supervisor stays connected during the
transition; its skill workers and the brain duplication will be
removed in a later plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 12: Update README and CLAUDE.md

Files:

Modify: README.md
Modify: CLAUDE.md (project-level)

Document the new MCP endpoint and the dual-MCP transitional state. Keep edits surgical — this is documentation, not a marketing rewrite.

Step 1: Update README architecture diagram and connect-a-project section

Read README.md first. Update the diagram to show two MCP endpoints. Update the "Connect a project" .mcp.json example to include both servers.

## Connect a project

Create `.mcp.json` in your project root:

```json
{
  "mcpServers": {
    "supervisor": {
      "type": "http",
      "url": "http://koala:30320/mcp"
    },
    "brain": {
      "type": "http",
      "url": "http://koala:30330/mcp"
    }
  }
}

Two MCP servers are exposed today, both reachable over Tailscale:

supervisor at koala:30320 — skill workers (tdd_*, review, debug, spec, retrospective, trainer, tier).
brain at koala:30330 — knowledge access (brain_query, brain_write, brain_ingest, brain_ingest_raw) and session_log.

The brain MCP is hosted by the ingestion service directly. The skill workers will move out of supervisor in a later plan.


- [ ] **Step 2: Add a brief note in CLAUDE.md about the dual MCP**

Append to the "Knowledge base access" section in `CLAUDE.md`:

```markdown
- **Brain MCP**: `http://koala:30330/mcp` — preferred path for brain_query, brain_write, brain_ingest_raw, brain_ingest, session_log.
- **Supervisor MCP**: `http://koala:30320/mcp` — skill workers (tdd, review, debug, spec, retrospective, trainer) until they migrate to SKILL.md.

Step 3: Commit

git add README.md CLAUDE.md
git commit -m "$(cat <<'EOF'
docs: document brain MCP endpoint at koala:30330

Updates the connect-a-project example and the CLAUDE.md knowledge
base section. Captures the transitional state where two MCPs coexist;
the supervisor MCP will shrink as skill workers move to SKILL.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"

Task 13: Final verification

Step 1: Re-run task check on the worktree

cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
task check 2>&1 | tail -20

Expected: lint clean, all tests pass, vet clean, govulncheck clean.

Step 2: Verify both MCPs respond from this Claude Code session

After user has restarted Claude Code:

/mcp lists supervisor (connected) and brain (connected).
Calling brain_query via the brain MCP returns results.
Calling tier via the supervisor MCP returns tier info.
Step 3: Verify brain/sessions/ has the smoke-test entry

kubectl --request-timeout=8s -n supervisor exec deploy/ingestion -- ls /app/brain/sessions/ 2>&1 | head -5

Expected: includes smoke-test.jsonl (from Task 10 step 4).

Step 4: Update plan checkbox status and merge

Mark all checkboxes complete in this plan file. Optionally squash the worktree branch into a single feature merge commit before merging to main:

cd /Users/mathias/Documents/local-dev/AI/hyperguild
git checkout main
git merge --no-ff feat/brain-mcp-migration -m "feat: brain MCP migration (extract brain + session_log into ingestion pod)"
git push origin main

Step 5: Clean up the worktree

Use superpowers:finishing-a-development-branch skill, or manually:

git worktree remove /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
git branch -d feat/brain-mcp-migration

Self-review

Spec coverage:

✅ MCP server skeleton (Task 1)
✅ session package (Task 2)
✅ brain_query (Task 3)
✅ brain_write with extracted helper (Task 4)
✅ brain_ingest_raw (Task 5)
✅ brain_ingest (Task 6)
✅ session_log (Task 7)
✅ Mount into ingestion server (Task 8)
✅ NodePort exposure (Task 9, separate repo)
✅ Live cluster verification (Task 10)
✅ .mcp.json switch (Task 11)
✅ Docs (Task 12)
✅ Final verification + cleanup (Task 13)

Placeholder scan: No "TBD", "implement later", or hand-wave error handling. Each step has actual code or actual commands.

Type consistency:

mcp.NewServer(brainDir string, *pipeline.Config, pipeline.CompleteFunc) — used consistently across Tasks 1, 3-7, and 8.
pipeline.RawPage shape — referenced in Task 5's test args, matches existing definition.
session.Entry shape — referenced in Task 7, matches Task 2's package definition.
api.WriteNote(brainDir, content, filename, type, domain) — defined Task 4, called from Task 4's MCP handler with positional args matching.

Dependencies on out-of-repo work:

Task 9 modifies the ~/dev/AI/infra repo. Pause point built in (Step 4 of Task 9).
Task 10 depends on CD completing (image build + Flux reconcile, ~2 min).

Reversibility:

All hyperguild changes are additive until Task 11. The .mcp.json change in Task 11 is one line, easily reverted.
The infra-repo NodePort can be reverted by deleting the new manifest line and re-pushing.
The supervisor MCP is untouched throughout. If brain MCP fails verification, fall back to supervisor MCP without code changes.

55 KiB Raw Blame History

Brain MCP Migration Implementation Plan

File Structure

Task 1: Add MCP server skeleton to ingestion

Task 2: Add session package to ingestion

Task 3: Implement brain_query MCP tool

Task 4: Extract WriteNote helper and implement brain_write

Task 5: Implement brain_ingest_raw

Task 6: Implement brain_ingest (LLM path)

Task 7: Implement session_log

Task 8: Wire MCP handler into ingestion server

Task 9: Add NodePort service in infra repo

Task 10: End-to-end smoke test against deployed brain MCP

Task 11: Update .mcp.json to use brain MCP

Task 12: Update README and CLAUDE.md

Task 13: Final verification

Self-review

55 KiB

Raw Blame History