From a412eee427750ef1c6ae15199b861aab153317b2 Mon Sep 17 00:00:00 2001 From: Mathias Bergqvist Date: Fri, 1 May 2026 09:27:28 +0200 Subject: [PATCH] docs: add brain MCP migration plan 13 TDD-disciplined tasks moving brain_* and session_log out of the supervisor pod and into the ingestion pod's MCP handler. Slice 1 of the larger SKILL.md + routing-MCP architecture migration. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../plans/2026-04-29-brain-mcp-migration.md | 1826 +++++++++++++++++ 1 file changed, 1826 insertions(+) create mode 100644 docs/superpowers/plans/2026-04-29-brain-mcp-migration.md diff --git a/docs/superpowers/plans/2026-04-29-brain-mcp-migration.md b/docs/superpowers/plans/2026-04-29-brain-mcp-migration.md new file mode 100644 index 0000000..661542d --- /dev/null +++ b/docs/superpowers/plans/2026-04-29-brain-mcp-migration.md @@ -0,0 +1,1826 @@ +# Brain MCP Migration Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Move the `brain_*` and `session_log` MCP tools out of the supervisor pod and into the ingestion pod, so Mode 1 (Claude Code, no supervisor) can reach the brain via direct HTTP MCP without the supervisor running. + +**Architecture:** A new `ingestion/internal/mcp` package adds a notification-aware MCP HTTP handler to the existing ingestion server, alongside its current REST endpoints. The handler dispatches `brain_query`, `brain_write`, `brain_ingest`, `brain_ingest_raw`, and `session_log` directly to existing pipeline/search/wiki packages within the same process — no HTTP round-trip to itself, no new module. A NodePort service exposes the MCP endpoint over Tailscale at `koala:30330/mcp`. The supervisor pod's brain skill remains in place during this slice; deletion is deferred to a later plan. + +**Tech Stack:** Go 1.24, net/http stdlib mux, encoding/json, testify, k3s manifests in the separate `infra` repo, Flux GitOps for deploy. + +--- + +## File Structure + +**New files (ingestion module):** +- `ingestion/internal/mcp/server.go` — JSON-RPC handler with notification skip +- `ingestion/internal/mcp/server_test.go` — covers parse, dispatch, notifications +- `ingestion/internal/mcp/handlers.go` — `brain_*` and `session_log` tool implementations +- `ingestion/internal/mcp/handlers_test.go` — covers each tool against a tmp brain dir +- `ingestion/internal/session/session.go` — copy of `internal/session/session.go` (will dedupe in supervisor-retirement plan) +- `ingestion/internal/session/session_test.go` — round-trip Append/Read + +**Modified files (ingestion module):** +- `ingestion/cmd/server/main.go:66-72` — register MCP handler at `POST /mcp` +- `ingestion/internal/api/handler.go` — extract pure `WriteNote` helper for reuse from MCP + +**New files (infra repo):** +- `infra/k3s/apps/supervisor/ingestion-nodeport.yaml` — exposes `ingestion:3300` as NodePort 30330 + +**Modified files (this repo, root):** +- `.mcp.json` — add `brain` server, leave `supervisor` until Mode 1 verified +- `README.md` — document the brain MCP endpoint and updated `.mcp.json` example +- `CLAUDE.md` — note the dual-MCP transitional state + +**Out of scope for this plan:** +- Removing the supervisor's brain skill (Plan 7) +- Removing `internal/session` from supervisor module (Plan 7) +- Adding `brain_search` (kb-retrieval not deployed; would be added with that integration) + +--- + +## Task 1: Add MCP server skeleton to ingestion + +**Files:** +- Create: `ingestion/internal/mcp/server.go` +- Create: `ingestion/internal/mcp/server_test.go` + +The MCP server in supervisor (`internal/mcp/server.go`) already has notification handling correct from prior work. We copy that shape — adapted to ingestion's module path — without pulling in supervisor's `registry` abstraction. Dispatch goes through a single `Server.handleCall` switch in this slice; if a third MCP server emerges later we can extract a registry then. + +- [ ] **Step 1: Write the failing tests** + +```go +// ingestion/internal/mcp/server_test.go +package mcp_test + +import ( + "bytes" + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/mathiasbq/hyperguild/ingestion/internal/mcp" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func body(t *testing.T, v any) *bytes.Buffer { + t.Helper() + b, err := json.Marshal(v) + require.NoError(t, err) + return bytes.NewBuffer(b) +} + +func TestServerInitialize(t *testing.T) { + srv := mcp.NewServer(t.TempDir(), nil, nil) + + req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{ + "jsonrpc": "2.0", "id": 1, "method": "initialize", + "params": map[string]any{}, + })) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + + assert.Equal(t, http.StatusOK, rr.Code) + var resp map[string]any + require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp)) + result := resp["result"].(map[string]any) + assert.Equal(t, "2024-11-05", result["protocolVersion"]) +} + +func TestServerToolsList(t *testing.T) { + srv := mcp.NewServer(t.TempDir(), nil, nil) + + req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{ + "jsonrpc": "2.0", "id": 2, "method": "tools/list", + })) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + + assert.Equal(t, http.StatusOK, rr.Code) + var resp map[string]any + require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp)) + tools := resp["result"].(map[string]any)["tools"].([]any) + names := make([]string, 0, len(tools)) + for _, t := range tools { + names = append(names, t.(map[string]any)["name"].(string)) + } + assert.ElementsMatch(t, []string{ + "brain_query", "brain_write", "brain_ingest_raw", "brain_ingest", "session_log", + }, names) +} + +func TestServerNotificationGetsNoBody(t *testing.T) { + srv := mcp.NewServer(t.TempDir(), nil, nil) + + req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{ + "jsonrpc": "2.0", "method": "notifications/initialized", + })) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + + assert.Equal(t, http.StatusOK, rr.Code) + assert.Empty(t, strings.TrimSpace(rr.Body.String())) +} + +func TestServerUnknownMethodReturnsError(t *testing.T) { + srv := mcp.NewServer(t.TempDir(), nil, nil) + + req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{ + "jsonrpc": "2.0", "id": 3, "method": "unknown/method", + })) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + + assert.Equal(t, http.StatusOK, rr.Code) + var resp map[string]any + require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp)) + assert.NotNil(t, resp["error"]) +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestServer' 2>&1 | head -30` +Expected: FAIL — package does not exist, `mcp.NewServer` undefined. + +- [ ] **Step 3: Write the minimal server implementation** + +```go +// ingestion/internal/mcp/server.go +// Package mcp implements an MCP HTTP handler for the ingestion service. +// Exposed tools: brain_query, brain_write, brain_ingest, brain_ingest_raw, session_log. +package mcp + +import ( + "context" + "encoding/json" + "net/http" + + "github.com/mathiasbq/hyperguild/ingestion/internal/pipeline" +) + +type request struct { + JSONRPC string `json:"jsonrpc"` + ID any `json:"id"` + Method string `json:"method"` + Params json.RawMessage `json:"params"` +} + +type response struct { + JSONRPC string `json:"jsonrpc"` + ID any `json:"id,omitempty"` + Result any `json:"result,omitempty"` + Error *rpcError `json:"error,omitempty"` +} + +type rpcError struct { + Code int `json:"code"` + Message string `json:"message"` +} + +// Server handles MCP JSON-RPC over HTTP for the ingestion service. +type Server struct { + brainDir string + pipeline pipeline.Config + llm pipeline.CompleteFunc // may be nil in tests +} + +// NewServer constructs a Server bound to brainDir. pipelineCfg supplies the +// LLM-backed pipeline; llm may be nil for non-LLM tools only. +func NewServer(brainDir string, pipelineCfg *pipeline.Config, llm pipeline.CompleteFunc) *Server { + cfg := pipeline.Config{} + if pipelineCfg != nil { + cfg = *pipelineCfg + } + return &Server{brainDir: brainDir, pipeline: cfg, llm: llm} +} + +func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { + var req request + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeError(w, nil, -32700, "parse error") + return + } + + // JSON-RPC 2.0 notifications (no id) must not receive a response. + if req.ID == nil { + return + } + + var result any + var rpcErr *rpcError + + switch req.Method { + case "initialize": + result = map[string]any{ + "protocolVersion": "2024-11-05", + "capabilities": map[string]any{"tools": map[string]any{}}, + "serverInfo": map[string]any{"name": "ingestion-brain", "version": "0.1.0"}, + } + case "tools/list": + result = map[string]any{"tools": s.tools()} + case "tools/call": + var p struct { + Name string `json:"name"` + Arguments json.RawMessage `json:"arguments"` + } + if err := json.Unmarshal(req.Params, &p); err != nil { + rpcErr = &rpcError{Code: -32602, Message: "invalid params"} + break + } + out, err := s.handleCall(r.Context(), p.Name, p.Arguments) + if err != nil { + rpcErr = &rpcError{Code: -32000, Message: err.Error()} + break + } + result = map[string]any{ + "content": []map[string]any{{"type": "text", "text": string(out)}}, + } + default: + rpcErr = &rpcError{Code: -32601, Message: "method not found: " + req.Method} + } + + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(response{ + JSONRPC: "2.0", + ID: req.ID, + Result: result, + Error: rpcErr, + }) +} + +func writeError(w http.ResponseWriter, id any, code int, msg string) { + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(response{ + JSONRPC: "2.0", + ID: id, + Error: &rpcError{Code: code, Message: msg}, + }) +} + +// handleCall dispatches a tools/call. Stub for Task 1; expanded in later tasks. +func (s *Server) handleCall(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error) { + return nil, &unknownToolError{name: name} +} + +type unknownToolError struct{ name string } + +func (e *unknownToolError) Error() string { return "unknown tool: " + e.name } +``` + +```go +// ingestion/internal/mcp/handlers.go (placeholder, filled in later tasks) +package mcp + +import "encoding/json" + +// tools returns the tool descriptors. Schemas are filled per-tool in subsequent tasks. +func (s *Server) tools() []map[string]any { + str := func(desc string) map[string]any { + return map[string]any{"type": "string", "description": desc} + } + int_ := func(desc string) map[string]any { + return map[string]any{"type": "integer", "description": desc} + } + schema := func(required []string, props map[string]any) json.RawMessage { + b, _ := json.Marshal(map[string]any{ + "type": "object", "required": required, "properties": props, + }) + return b + } + + return []map[string]any{ + { + "name": "brain_query", + "description": "BM25 full-text search across brain/knowledge/ and brain/wiki/ markdown files.", + "inputSchema": schema([]string{"query"}, map[string]any{ + "query": str("search terms"), + "limit": int_("max results, default 5"), + }), + }, + { + "name": "brain_write", + "description": "Write a raw knowledge note to brain/knowledge/.", + "inputSchema": schema([]string{"content"}, map[string]any{ + "content": str("markdown content"), + "filename": str("optional filename"), + "type": str("optional frontmatter type"), + "domain": str("optional frontmatter domain"), + }), + }, + { + "name": "brain_ingest_raw", + "description": "Ingest pre-structured pages into the brain wiki, bypassing the LLM extraction step.", + "inputSchema": schema([]string{"source", "pages"}, map[string]any{ + "source": str("source name"), + "pages": map[string]any{"type": "array"}, + "dry_run": map[string]any{"type": "boolean"}, + }), + }, + { + "name": "brain_ingest", + "description": "Ingest content into the brain wiki via the LLM extraction pipeline.", + "inputSchema": schema([]string{}, map[string]any{ + "content": str("raw content; required when path is empty"), + "source": str("source name; required when path is empty"), + "path": str("file path; mutually exclusive with content+source"), + "dry_run": map[string]any{"type": "boolean"}, + }), + }, + { + "name": "session_log", + "description": "Append a structured entry to brain/sessions/.jsonl.", + "inputSchema": schema([]string{"session_id"}, map[string]any{ + "session_id": str("session identifier"), + "skill": str("skill name"), + "phase": str("phase within the skill"), + "project_root": str("absolute project root"), + "final_status": str("ok | error | skipped"), + "file_path": str("optional file produced"), + "model_used": str("optional model identifier"), + "duration_ms": int_("optional duration in ms"), + "message": str("optional free-text"), + }), + }, + } +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestServer' 2>&1 | tail -20` +Expected: All four `TestServer*` tests PASS. + +- [ ] **Step 5: Commit** + +```bash +cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration +git add ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server_test.go +git commit -m "$(cat <<'EOF' +feat(ingestion): add MCP server skeleton with tools/list + +Adds an MCP HTTP handler under ingestion/internal/mcp. Implements +initialize, tools/list, and the JSON-RPC notification skip from prior +work. Tool dispatch is stubbed (returns unknown-tool error) and will be +filled in by subsequent tasks. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 2: Add session package to ingestion + +**Files:** +- Create: `ingestion/internal/session/session.go` +- Create: `ingestion/internal/session/session_test.go` + +We deliberately copy `internal/session/session.go` from supervisor rather than introduce a third Go module. Plan 7 (supervisor retirement) will delete the supervisor copy. Until then both exist; they will not drift because no edits happen during transition. + +- [ ] **Step 1: Write the failing test** + +```go +// ingestion/internal/session/session_test.go +package session_test + +import ( + "path/filepath" + "testing" + "time" + + "github.com/mathiasbq/hyperguild/ingestion/internal/session" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestAppendAndRead(t *testing.T) { + dir := t.TempDir() + sid := "test-session" + + e1 := session.Entry{ + SessionID: sid, + Timestamp: time.Now().UTC().Truncate(time.Second), + Skill: "tdd", + Phase: "red", + FinalStatus: "ok", + } + e2 := session.Entry{ + SessionID: sid, + Timestamp: time.Now().UTC().Truncate(time.Second), + Skill: "tdd", + Phase: "green", + FinalStatus: "ok", + } + + require.NoError(t, session.Append(dir, sid, e1)) + require.NoError(t, session.Append(dir, sid, e2)) + + got, err := session.Read(dir, sid) + require.NoError(t, err) + require.Len(t, got, 2) + assert.Equal(t, "red", got[0].Phase) + assert.Equal(t, "green", got[1].Phase) + + // File path is sessionsDir/.jsonl + _, statErr := filepath.Abs(filepath.Join(dir, sid+".jsonl")) + require.NoError(t, statErr) +} + +func TestReadMissingReturnsEmpty(t *testing.T) { + got, err := session.Read(t.TempDir(), "nope") + require.NoError(t, err) + assert.Empty(t, got) +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd ingestion && go test ./internal/session/... -v 2>&1 | head -20` +Expected: FAIL — package does not exist. + +- [ ] **Step 3: Write the implementation** + +Copy `/Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration/internal/session/session.go` verbatim into `ingestion/internal/session/session.go`, changing only the package comment header to reference the ingestion path. + +```go +// ingestion/internal/session/session.go +package session + +import ( + "bufio" + "encoding/json" + "errors" + "fmt" + "io/fs" + "os" + "path/filepath" + "time" +) + +// Entry is one skill invocation record, appended to the session JSONL log. +type Entry struct { + SessionID string `json:"session_id"` + Timestamp time.Time `json:"timestamp"` + Skill string `json:"skill"` + Phase string `json:"phase,omitempty"` + ProjectRoot string `json:"project_root,omitempty"` + Input json.RawMessage `json:"input,omitempty"` + Attempts []Attempt `json:"attempts,omitempty"` + FinalStatus string `json:"final_status"` + FilePath string `json:"file_path,omitempty"` + ModelUsed string `json:"model_used,omitempty"` + DurationMs int64 `json:"duration_ms,omitempty"` + Message string `json:"message,omitempty"` +} + +// Attempt represents one subprocess invocation within a skill call. +type Attempt struct { + Attempt int `json:"attempt"` + Model string `json:"model"` + Tier string `json:"tier"` + DurationMs int64 `json:"duration_ms"` + WarmStart bool `json:"warm_start"` + Verified bool `json:"verified"` + Verdict string `json:"verdict,omitempty"` + Feedback string `json:"feedback,omitempty"` + OutputSummary string `json:"output_summary,omitempty"` + RunnerOutput string `json:"runner_output,omitempty"` +} + +// Append writes entry as a single JSON line to sessionsDir/{sessionID}.jsonl. +func Append(sessionsDir, sessionID string, entry Entry) error { + if err := os.MkdirAll(sessionsDir, 0o755); err != nil { + return fmt.Errorf("create sessions dir: %w", err) + } + path := filepath.Join(sessionsDir, sessionID+".jsonl") + f, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644) + if err != nil { + return fmt.Errorf("open session log: %w", err) + } + line, err := json.Marshal(entry) + if err != nil { + _ = f.Close() + return fmt.Errorf("marshal entry: %w", err) + } + if _, err = fmt.Fprintf(f, "%s\n", line); err != nil { + _ = f.Close() + return fmt.Errorf("write entry: %w", err) + } + if err = f.Close(); err != nil { + return fmt.Errorf("close session log: %w", err) + } + return nil +} + +// Read returns all entries for sessionID. Returns empty slice if no log exists. +func Read(sessionsDir, sessionID string) ([]Entry, error) { + path := filepath.Join(sessionsDir, sessionID+".jsonl") + f, err := os.Open(path) + if errors.Is(err, fs.ErrNotExist) { + return []Entry{}, nil + } + if err != nil { + return nil, fmt.Errorf("open session log: %w", err) + } + defer f.Close() //nolint:errcheck + + var entries []Entry + scanner := bufio.NewScanner(f) + scanner.Buffer(make([]byte, 0, 256*1024), 1<<20) + for scanner.Scan() { + line := scanner.Bytes() + if len(line) == 0 { + continue + } + var e Entry + if err := json.Unmarshal(line, &e); err != nil { + return nil, fmt.Errorf("parse entry: %w", err) + } + entries = append(entries, e) + } + return entries, scanner.Err() +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd ingestion && go test ./internal/session/... -v 2>&1 | tail -10` +Expected: Both tests PASS. + +- [ ] **Step 5: Commit** + +```bash +git add ingestion/internal/session +git commit -m "$(cat <<'EOF' +feat(ingestion): add session package for JSONL log persistence + +Copy of internal/session from the supervisor module — the ingestion +service needs it for the upcoming session_log MCP tool. The supervisor +copy will be removed in the supervisor-retirement plan; until then +the two packages are intentionally identical and pinned (no edits). + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 3: Implement brain_query MCP tool + +**Files:** +- Modify: `ingestion/internal/mcp/handlers.go` (add `brainQuery` method + dispatch) +- Modify: `ingestion/internal/mcp/server.go` (wire `handleCall` to call `brainQuery`) +- Create test in: `ingestion/internal/mcp/handlers_test.go` + +`brain_query` calls the existing pure function `search.Query(brainDir, query, limit)`. No HTTP self-call needed. + +- [ ] **Step 1: Write the failing test** + +```go +// ingestion/internal/mcp/handlers_test.go +package mcp_test + +import ( + "bytes" + "encoding/json" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "testing" + + "github.com/mathiasbq/hyperguild/ingestion/internal/mcp" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func toolCall(t *testing.T, srv http.Handler, name string, args map[string]any) map[string]any { + t.Helper() + bodyBytes, err := json.Marshal(map[string]any{ + "jsonrpc": "2.0", "id": 1, "method": "tools/call", + "params": map[string]any{"name": name, "arguments": args}, + }) + require.NoError(t, err) + req := httptest.NewRequest(http.MethodPost, "/mcp", bytes.NewReader(bodyBytes)) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + require.Equal(t, http.StatusOK, rr.Code) + var resp map[string]any + require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp)) + return resp +} + +func TestBrainQueryReturnsResults(t *testing.T) { + brainDir := t.TempDir() + knowledge := filepath.Join(brainDir, "knowledge") + require.NoError(t, os.MkdirAll(knowledge, 0o755)) + require.NoError(t, os.WriteFile( + filepath.Join(knowledge, "tdd.md"), + []byte("# TDD\n\nTest-driven development is iterative.\n"), + 0o644, + )) + + srv := mcp.NewServer(brainDir, nil, nil) + resp := toolCall(t, srv, "brain_query", map[string]any{"query": "tdd"}) + + require.Nil(t, resp["error"]) + result := resp["result"].(map[string]any) + content := result["content"].([]any) + require.NotEmpty(t, content) + text := content[0].(map[string]any)["text"].(string) + assert.Contains(t, text, "tdd.md") +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainQuery' 2>&1 | head -20` +Expected: FAIL — `unknown tool: brain_query`. + +- [ ] **Step 3: Implement brainQuery** + +Add to `ingestion/internal/mcp/handlers.go`: + +```go +// (append to existing handlers.go) + +import ( + "context" + "encoding/json" + "fmt" + + "github.com/mathiasbq/hyperguild/ingestion/internal/search" +) + +type brainQueryArgs struct { + Query string `json:"query"` + Limit int `json:"limit,omitempty"` +} + +func (s *Server) brainQuery(ctx context.Context, args json.RawMessage) (json.RawMessage, error) { + var a brainQueryArgs + if err := json.Unmarshal(args, &a); err != nil { + return nil, fmt.Errorf("parse args: %w", err) + } + if a.Query == "" { + return nil, fmt.Errorf("query is required") + } + if a.Limit == 0 { + a.Limit = 5 + } + results, err := search.Query(s.brainDir, a.Query, a.Limit) + if err != nil { + return nil, fmt.Errorf("search: %w", err) + } + return json.Marshal(map[string]any{"results": results}) +} +``` + +Update `ingestion/internal/mcp/server.go`'s `handleCall` to dispatch: + +```go +// ingestion/internal/mcp/server.go — replace handleCall and unknownToolError block + +func (s *Server) handleCall(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error) { + switch name { + case "brain_query": + return s.brainQuery(ctx, args) + default: + return nil, fmt.Errorf("unknown tool: %s", name) + } +} +``` + +Add the `"fmt"` import to `server.go` and remove the now-unused `unknownToolError` type and its `Error()` method. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainQuery' 2>&1 | tail -10` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/handlers_test.go ingestion/internal/mcp/server.go +git commit -m "$(cat <<'EOF' +feat(ingestion): implement brain_query MCP tool + +Wraps the existing search.Query function. Same BM25 over +brain/knowledge/ and brain/wiki/ that the HTTP /query endpoint serves. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 4: Extract WriteNote helper and implement brain_write + +**Files:** +- Modify: `ingestion/internal/api/handler.go` (extract `WriteNote` from `Handler.Write`) +- Modify: `ingestion/internal/mcp/handlers.go` (call `api.WriteNote`) +- Modify: `ingestion/internal/mcp/server.go` (dispatch `brain_write`) +- Modify: `ingestion/internal/api/handler_test.go` (regression: existing /write still works) +- Modify: `ingestion/internal/mcp/handlers_test.go` (new: brain_write writes a file) + +The current `Handler.Write` mixes JSON decoding, validation, frontmatter construction, path safety, and disk writing. We extract a pure `WriteNote` helper so MCP and HTTP both call it. + +- [ ] **Step 1: Write the failing tests** + +Add to `ingestion/internal/mcp/handlers_test.go`: + +```go +func TestBrainWriteCreatesFile(t *testing.T) { + brainDir := t.TempDir() + srv := mcp.NewServer(brainDir, nil, nil) + + resp := toolCall(t, srv, "brain_write", map[string]any{ + "content": "# Test\n\nbody", + "filename": "test.md", + "type": "note", + "domain": "personal", + }) + require.Nil(t, resp["error"]) + + // File should exist under brain/knowledge/test.md with frontmatter + got, err := os.ReadFile(filepath.Join(brainDir, "knowledge", "test.md")) + require.NoError(t, err) + assert.Contains(t, string(got), "type: note") + assert.Contains(t, string(got), "domain: personal") + assert.Contains(t, string(got), "# Test") +} + +func TestBrainWriteRejectsTraversal(t *testing.T) { + brainDir := t.TempDir() + srv := mcp.NewServer(brainDir, nil, nil) + + resp := toolCall(t, srv, "brain_write", map[string]any{ + "content": "x", + "filename": "../escape.md", + }) + assert.NotNil(t, resp["error"]) +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainWrite' 2>&1 | head -20` +Expected: FAIL — `unknown tool: brain_write`. + +- [ ] **Step 3: Extract WriteNote in api/handler.go** + +Replace the body of `Handler.Write` with a thin wrapper that calls a new exported function `WriteNote`. Read `ingestion/internal/api/handler.go:88-142` first to confirm the exact code. + +```go +// ingestion/internal/api/handler.go — add new exported function before func (h *Handler) Write +// and replace the body of Write to call WriteNote. + +// WriteNote writes a markdown file to brainDir/knowledge/, optionally +// prefixed with YAML frontmatter built from typ and domain. Returns the path +// relative to brainDir (forward-slashed). Filename traversal is rejected. +func WriteNote(brainDir, content, filename, typ, domain string) (string, error) { + if content == "" { + return "", fmt.Errorf("content is required") + } + if filename == "" { + filename = fmt.Sprintf("%s-auto.md", time.Now().UTC().Format("2006-01-02-150405")) + } + + rawDir := filepath.Join(brainDir, "knowledge") + if err := os.MkdirAll(rawDir, 0o755); err != nil { + return "", fmt.Errorf("create raw dir: %w", err) + } + + finalContent := content + if typ != "" || domain != "" { + var fm strings.Builder + fm.WriteString("---\n") + if typ != "" { + fmt.Fprintf(&fm, "type: %s\n", typ) + } + if domain != "" { + fmt.Fprintf(&fm, "domain: %s\n", domain) + } + fm.WriteString("---\n") + finalContent = fm.String() + content + } + + base := filepath.Base(filename) + if !strings.HasSuffix(base, ".md") { + base += ".md" + } + dest := filepath.Join(rawDir, base) + if !strings.HasPrefix(filepath.Clean(dest)+string(os.PathSeparator), + filepath.Clean(rawDir)+string(os.PathSeparator)) { + return "", fmt.Errorf("invalid filename") + } + if err := os.WriteFile(dest, []byte(finalContent), 0o644); err != nil { + return "", fmt.Errorf("write: %w", err) + } + + rel, _ := filepath.Rel(brainDir, dest) + return filepath.ToSlash(rel), nil +} +``` + +Replace `Handler.Write` body to call `WriteNote`: + +```go +func (h *Handler) Write(w http.ResponseWriter, r *http.Request) { + var req writeRequest + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeError(w, http.StatusBadRequest, "invalid JSON") + return + } + relPath, err := WriteNote(h.brainDir, req.Content, req.Filename, req.Type, req.Domain) + if err != nil { + h.logger.Error("write failed", "err", err) + writeError(w, http.StatusBadRequest, err.Error()) + return + } + writeJSON(w, map[string]string{"path": relPath}) +} +``` + +- [ ] **Step 4: Add brainWrite handler and dispatch** + +Append to `ingestion/internal/mcp/handlers.go`: + +```go +import ( + // add to existing import block + "github.com/mathiasbq/hyperguild/ingestion/internal/api" +) + +type brainWriteArgs struct { + Content string `json:"content"` + Filename string `json:"filename,omitempty"` + Type string `json:"type,omitempty"` + Domain string `json:"domain,omitempty"` +} + +func (s *Server) brainWrite(ctx context.Context, args json.RawMessage) (json.RawMessage, error) { + var a brainWriteArgs + if err := json.Unmarshal(args, &a); err != nil { + return nil, fmt.Errorf("parse args: %w", err) + } + relPath, err := api.WriteNote(s.brainDir, a.Content, a.Filename, a.Type, a.Domain) + if err != nil { + return nil, err + } + return json.Marshal(map[string]string{"path": relPath}) +} +``` + +Update `handleCall` switch: + +```go +case "brain_write": + return s.brainWrite(ctx, args) +``` + +- [ ] **Step 5: Run all ingestion tests** + +Run: `cd ingestion && go test -count=1 ./... 2>&1 | tail -15` +Expected: All packages PASS, including the existing `api` tests (regression: HTTP /write still works) and the new `mcp` tests. + +- [ ] **Step 6: Commit** + +```bash +git add ingestion/internal/api/handler.go ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go +git commit -m "$(cat <<'EOF' +feat(ingestion): extract WriteNote helper and add brain_write MCP tool + +api.WriteNote captures the file-write logic that was previously inline +in Handler.Write. The existing HTTP endpoint now delegates to it; the +new MCP brain_write tool reuses the same function. Path-traversal +guard is preserved. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 5: Implement brain_ingest_raw + +**Files:** +- Modify: `ingestion/internal/mcp/handlers.go` (add `brainIngestRaw`) +- Modify: `ingestion/internal/mcp/server.go` (dispatch) +- Modify: `ingestion/internal/mcp/handlers_test.go` (smoke test against tmp brain) + +`pipeline.RunRaw(brainDir, source, pages, dryRun)` is already pure — direct call. + +- [ ] **Step 1: Write the failing test** + +Add to `ingestion/internal/mcp/handlers_test.go`: + +```go +import ( + // existing + add: + "github.com/mathiasbq/hyperguild/ingestion/internal/pipeline" +) + +func TestBrainIngestRawDryRun(t *testing.T) { + brainDir := t.TempDir() + require.NoError(t, os.MkdirAll(filepath.Join(brainDir, "wiki", "concepts"), 0o755)) + srv := mcp.NewServer(brainDir, nil, nil) + + resp := toolCall(t, srv, "brain_ingest_raw", map[string]any{ + "source": "test-source", + "dry_run": true, + "pages": []map[string]any{ + { + "title": "Test Concept", + "type": "concept", + "content": "## Definition\nA test concept.", + }, + }, + }) + require.Nil(t, resp["error"]) + result := resp["result"].(map[string]any) + content := result["content"].([]any) + text := content[0].(map[string]any)["text"].(string) + + var parsed struct { + Pages []string `json:"pages"` + } + require.NoError(t, json.Unmarshal([]byte(text), &parsed)) + assert.Contains(t, parsed.Pages[0], "wiki/concepts/test-concept.md") + + // dry_run: no file should exist + _, err := os.Stat(filepath.Join(brainDir, "wiki", "concepts", "test-concept.md")) + assert.True(t, os.IsNotExist(err)) +} + +var _ = pipeline.RawPage{} // keep import linked for future tests +``` + +- [ ] **Step 2: Run test, verify failure** + +Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainIngestRaw' 2>&1 | head -20` +Expected: FAIL — `unknown tool: brain_ingest_raw`. + +- [ ] **Step 3: Implement** + +Append to `ingestion/internal/mcp/handlers.go`: + +```go +type brainIngestRawArgs struct { + Source string `json:"source"` + Pages []pipeline.RawPage `json:"pages"` + DryRun bool `json:"dry_run,omitempty"` +} + +func (s *Server) brainIngestRaw(ctx context.Context, args json.RawMessage) (json.RawMessage, error) { + var a brainIngestRawArgs + if err := json.Unmarshal(args, &a); err != nil { + return nil, fmt.Errorf("parse args: %w", err) + } + if a.Source == "" { + return nil, fmt.Errorf("source is required") + } + if len(a.Pages) == 0 { + return nil, fmt.Errorf("pages must be non-empty") + } + result, err := pipeline.RunRaw(s.brainDir, a.Source, a.Pages, a.DryRun) + if err != nil { + return nil, fmt.Errorf("ingest: %w", err) + } + pages := result.Pages + if pages == nil { + pages = []string{} + } + warnings := result.Warnings + if warnings == nil { + warnings = []string{} + } + return json.Marshal(map[string]any{"pages": pages, "warnings": warnings}) +} +``` + +Update import block to include `pipeline` package. + +Update `handleCall` switch: + +```go +case "brain_ingest_raw": + return s.brainIngestRaw(ctx, args) +``` + +- [ ] **Step 4: Run tests** + +Run: `cd ingestion && go test -count=1 ./internal/mcp/... 2>&1 | tail -10` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go +git commit -m "$(cat <<'EOF' +feat(ingestion): implement brain_ingest_raw MCP tool + +Wraps pipeline.RunRaw directly. Same dry-run semantics as the HTTP +/ingest-raw endpoint. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 6: Implement brain_ingest (LLM path) + +**Files:** +- Modify: `ingestion/internal/mcp/handlers.go` (add `brainIngest`) +- Modify: `ingestion/internal/mcp/server.go` (dispatch + LLM availability check) +- Modify: `ingestion/internal/mcp/handlers_test.go` (path-vs-content+source mutex test) + +`brain_ingest` takes either `path` (file) or `content+source`. Calls `pipeline.Run` which uses the LLM. The MCP server's `s.pipeline` config holds the `Complete` function. + +- [ ] **Step 1: Write the failing tests** + +Add to `ingestion/internal/mcp/handlers_test.go`: + +```go +func TestBrainIngestRejectsBoth(t *testing.T) { + brainDir := t.TempDir() + srv := mcp.NewServer(brainDir, nil, nil) + + resp := toolCall(t, srv, "brain_ingest", map[string]any{ + "content": "x", + "source": "y", + "path": "/tmp/foo.md", + }) + assert.NotNil(t, resp["error"]) +} + +func TestBrainIngestRequiresOne(t *testing.T) { + brainDir := t.TempDir() + srv := mcp.NewServer(brainDir, nil, nil) + + resp := toolCall(t, srv, "brain_ingest", map[string]any{}) + assert.NotNil(t, resp["error"]) +} +``` + +- [ ] **Step 2: Run, verify FAIL** + +Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainIngest' 2>&1 | head -20` +Expected: FAIL — `unknown tool: brain_ingest`. + +- [ ] **Step 3: Implement** + +Append to `ingestion/internal/mcp/handlers.go`: + +```go +import ( + // add: + "github.com/mathiasbq/hyperguild/ingestion/internal/extract" + "path/filepath" + "strings" +) + +type brainIngestArgs struct { + Content string `json:"content,omitempty"` + Source string `json:"source,omitempty"` + Path string `json:"path,omitempty"` + DryRun bool `json:"dry_run,omitempty"` +} + +func (s *Server) brainIngest(ctx context.Context, args json.RawMessage) (json.RawMessage, error) { + var a brainIngestArgs + if err := json.Unmarshal(args, &a); err != nil { + return nil, fmt.Errorf("parse args: %w", err) + } + if a.Path != "" && a.Content != "" { + return nil, fmt.Errorf("path and content+source are mutually exclusive") + } + if a.Path == "" && a.Content == "" { + return nil, fmt.Errorf("either path or content+source is required") + } + if s.pipeline.Complete == nil { + return nil, fmt.Errorf("LLM not configured: set INGEST_LLM_URL") + } + + if a.Path != "" { + text, err := extract.Text(a.Path) + if err != nil { + return nil, fmt.Errorf("extract: %w", err) + } + source := a.Source + if source == "" { + source = filepath.Base(strings.TrimSuffix(a.Path, filepath.Ext(a.Path))) + } + return s.runIngest(ctx, text, source, a.DryRun) + } + if a.Source == "" { + return nil, fmt.Errorf("source is required when content is provided") + } + return s.runIngest(ctx, a.Content, a.Source, a.DryRun) +} + +func (s *Server) runIngest(ctx context.Context, content, source string, dryRun bool) (json.RawMessage, error) { + result, err := pipeline.Run(ctx, s.pipeline, s.brainDir, content, source, dryRun) + if err != nil { + return nil, fmt.Errorf("ingest: %w", err) + } + pages := result.Pages + if pages == nil { + pages = []string{} + } + warnings := result.Warnings + if warnings == nil { + warnings = []string{} + } + return json.Marshal(map[string]any{"pages": pages, "warnings": warnings}) +} +``` + +Update `handleCall` switch: + +```go +case "brain_ingest": + return s.brainIngest(ctx, args) +``` + +- [ ] **Step 4: Run tests** + +Run: `cd ingestion && go test -count=1 ./internal/mcp/... 2>&1 | tail -10` +Expected: All PASS. + +- [ ] **Step 5: Commit** + +```bash +git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go +git commit -m "$(cat <<'EOF' +feat(ingestion): implement brain_ingest MCP tool + +Wraps pipeline.Run with the existing LLM client. Mirrors the HTTP +/ingest and /ingest-path semantics — accepts either path or +content+source, validates mutual exclusion, surfaces an explicit error +when the LLM client is not configured (test-mode). + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 7: Implement session_log + +**Files:** +- Modify: `ingestion/internal/mcp/handlers.go` (add `sessionLog`) +- Modify: `ingestion/internal/mcp/server.go` (dispatch) +- Modify: `ingestion/internal/mcp/handlers_test.go` (verify JSONL line written) + +Writes to `${BRAIN_DIR}/sessions/.jsonl` via the new `ingestion/internal/session` package. + +- [ ] **Step 1: Write the failing test** + +Add to `ingestion/internal/mcp/handlers_test.go`: + +```go +func TestSessionLogAppends(t *testing.T) { + brainDir := t.TempDir() + srv := mcp.NewServer(brainDir, nil, nil) + + resp := toolCall(t, srv, "session_log", map[string]any{ + "session_id": "session-x", + "skill": "tdd", + "phase": "red", + "final_status": "ok", + }) + require.Nil(t, resp["error"]) + + got, err := os.ReadFile(filepath.Join(brainDir, "sessions", "session-x.jsonl")) + require.NoError(t, err) + assert.Contains(t, string(got), `"skill":"tdd"`) + assert.Contains(t, string(got), `"phase":"red"`) +} + +func TestSessionLogRequiresSessionID(t *testing.T) { + srv := mcp.NewServer(t.TempDir(), nil, nil) + resp := toolCall(t, srv, "session_log", map[string]any{"skill": "tdd"}) + assert.NotNil(t, resp["error"]) +} +``` + +- [ ] **Step 2: Run, verify FAIL** + +Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestSessionLog' 2>&1 | head -20` +Expected: FAIL. + +- [ ] **Step 3: Implement** + +Append to `ingestion/internal/mcp/handlers.go`: + +```go +import ( + // add: + "path/filepath" + "time" + "github.com/mathiasbq/hyperguild/ingestion/internal/session" +) + +type sessionLogArgs struct { + SessionID string `json:"session_id"` + Skill string `json:"skill,omitempty"` + Phase string `json:"phase,omitempty"` + ProjectRoot string `json:"project_root,omitempty"` + FinalStatus string `json:"final_status,omitempty"` + FilePath string `json:"file_path,omitempty"` + ModelUsed string `json:"model_used,omitempty"` + DurationMs int64 `json:"duration_ms,omitempty"` + Message string `json:"message,omitempty"` +} + +func (s *Server) sessionLog(ctx context.Context, args json.RawMessage) (json.RawMessage, error) { + var a sessionLogArgs + if err := json.Unmarshal(args, &a); err != nil { + return nil, fmt.Errorf("parse args: %w", err) + } + if a.SessionID == "" { + return nil, fmt.Errorf("session_id is required") + } + entry := session.Entry{ + SessionID: a.SessionID, + Timestamp: time.Now().UTC(), + Skill: a.Skill, + Phase: a.Phase, + ProjectRoot: a.ProjectRoot, + FinalStatus: a.FinalStatus, + FilePath: a.FilePath, + ModelUsed: a.ModelUsed, + DurationMs: a.DurationMs, + Message: a.Message, + } + dir := filepath.Join(s.brainDir, "sessions") + if err := session.Append(dir, a.SessionID, entry); err != nil { + return nil, fmt.Errorf("append: %w", err) + } + return json.Marshal(map[string]string{"status": "ok", "session_id": a.SessionID}) +} +``` + +Update `handleCall`: + +```go +case "session_log": + return s.sessionLog(ctx, args) +``` + +- [ ] **Step 4: Run tests** + +Run: `cd ingestion && go test -count=1 ./internal/mcp/... 2>&1 | tail -10` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go +git commit -m "$(cat <<'EOF' +feat(ingestion): implement session_log MCP tool + +Appends a JSON line to brainDir/sessions/.jsonl using the +copied session package. Required for upcoming pass-rate logging. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 8: Wire MCP handler into ingestion server + +**Files:** +- Modify: `ingestion/cmd/server/main.go:66-72` (mount MCP at POST /mcp) + +- [ ] **Step 1: Write a smoke test that exercises the live handler over HTTP** + +Add a new file `ingestion/internal/mcp/integration_test.go`: + +```go +package mcp_test + +import ( + "bytes" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "testing" + + "github.com/mathiasbq/hyperguild/ingestion/internal/mcp" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestMCPMountedHandler(t *testing.T) { + srv := mcp.NewServer(t.TempDir(), nil, nil) + mux := http.NewServeMux() + mux.Handle("POST /mcp", srv) + + ts := httptest.NewServer(mux) + defer ts.Close() + + body, _ := json.Marshal(map[string]any{ + "jsonrpc": "2.0", "id": 1, "method": "tools/list", + }) + resp, err := http.Post(ts.URL+"/mcp", "application/json", bytes.NewReader(body)) + require.NoError(t, err) + defer resp.Body.Close() + assert.Equal(t, http.StatusOK, resp.StatusCode) + + out, _ := io.ReadAll(resp.Body) + assert.Contains(t, string(out), `"brain_query"`) +} +``` + +- [ ] **Step 2: Run, expect FAIL until step 3** + +This test passes today (it constructs its own mux). Skip step 2 — the test verifies the wiring shape; the actual wiring change in step 3 ensures `cmd/server/main.go` also mounts it. + +Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestMCPMountedHandler' 2>&1 | tail -10` +Expected: PASS (test is module-local and constructs its own mux). + +- [ ] **Step 3: Wire into main.go** + +Modify `ingestion/cmd/server/main.go`: + +Add to imports: +```go +"github.com/mathiasbq/hyperguild/ingestion/internal/mcp" +``` + +After `h := api.NewHandler(...)` and before the `mux := http.NewServeMux()` block, construct the MCP server: + +```go +mcpSrv := mcp.NewServer(brainDir, &pipelineCfg, llmClient.Complete) +``` + +Add the route after the existing POST handlers (around line 72): + +```go +mux.Handle("POST /mcp", mcpSrv) +``` + +Update the startup log line to mention MCP: + +```go +logger.Info("ingestion server starting", + "addr", addr, + "brain_dir", brainDir, + "llm_url", llmURL, + "llm_model", llmModel, + "chunk_size", chunkSize, + "watch_interval", watchIntervalLog, + "mcp_enabled", true, +) +``` + +- [ ] **Step 4: Build and run locally as a smoke test** + +```bash +cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration +mkdir -p /tmp/brain-mcp-test/{wiki,sessions,knowledge} +INGEST_BRAIN_DIR=/tmp/brain-mcp-test \ +INGEST_PORT=33301 \ +INGEST_WATCH_INTERVAL=0 \ +go run ./ingestion/cmd/server/ & +SERVER_PID=$! +sleep 2 + +# tools/list +curl -sS -X POST http://localhost:33301/mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | head -c 500 +echo "" + +# notifications/initialized → empty body +curl -sS -i -X POST http://localhost:33301/mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","method":"notifications/initialized"}' | head -10 +echo "" + +# brain_query smoke +curl -sS -X POST http://localhost:33301/mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"brain_query","arguments":{"query":"x"}}}' \ + | head -c 200 +echo "" + +kill $SERVER_PID +wait $SERVER_PID 2>/dev/null +rm -rf /tmp/brain-mcp-test +``` + +Expected: +- tools/list returns JSON with all 5 brain tool names. +- notifications/initialized returns HTTP/1.1 200 with `Content-Length: 0`. +- brain_query returns `{"results":null}` (empty brain). + +- [ ] **Step 5: Run full ingestion test suite** + +Run: `cd ingestion && go test -race -count=1 ./... 2>&1 | tail -15` +Expected: All packages PASS. + +- [ ] **Step 6: Commit** + +```bash +git add ingestion/cmd/server/main.go ingestion/internal/mcp/integration_test.go +git commit -m "$(cat <<'EOF' +feat(ingestion): mount MCP handler at POST /mcp + +The ingestion server now exposes both REST and MCP on the same port +(3300). MCP shares brainDir, pipeline config, and LLM client with the +REST handlers — single source of process state. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 9: Add NodePort service in infra repo + +**Files (in `~/Documents/local-dev/AI/infra`):** +- Create: `infra/k3s/apps/supervisor/ingestion-nodeport.yaml` +- Modify: `infra/k3s/apps/supervisor/kustomization.yaml` + +This is in a different repo. The branch and PR happen there. **Confirm with the user before pushing** — `infra` repo changes touch live cluster routing. + +- [ ] **Step 1: Create the NodePort manifest** + +Read `infra/k3s/apps/supervisor/supervisor-nodeport.yaml` first to mirror its style. + +```yaml +# infra/k3s/apps/supervisor/ingestion-nodeport.yaml +apiVersion: v1 +kind: Service +metadata: + name: ingestion-nodeport + namespace: supervisor +spec: + type: NodePort + selector: + app: ingestion + ports: + - name: http + port: 3300 + targetPort: 3300 + nodePort: 30330 +``` + +- [ ] **Step 2: Add to kustomization.yaml** + +Modify `infra/k3s/apps/supervisor/kustomization.yaml` resources list: + +```yaml +resources: + - namespace.yaml + - deployment.yaml + - service.yaml + - secrets.enc.yaml + - supervisor-nodeport.yaml + - ingestion-deployment.yaml + - ingestion-service.yaml + - ingestion-nodeport.yaml # added +``` + +- [ ] **Step 3: Commit (in infra repo)** + +```bash +cd /Users/mathias/Documents/local-dev/AI/infra +git checkout -b feat/ingestion-nodeport +git add k3s/apps/supervisor/ingestion-nodeport.yaml k3s/apps/supervisor/kustomization.yaml +git commit -m "$(cat <<'EOF' +feat(supervisor): expose ingestion as NodePort 30330 for direct MCP + +Pairs with hyperguild's brain MCP migration — Claude Code can now +reach brain MCP at http://koala:30330/mcp without going through the +supervisor pod. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +- [ ] **Step 4: Confirm with user before push** + +Pause and ask: "Push the infra change to gitea origin? Flux will reconcile within ~30s and add the NodePort service. Confirm?" + +If confirmed: + +```bash +cd /Users/mathias/Documents/local-dev/AI/infra +git push origin feat/ingestion-nodeport +``` + +(Or merge to main per your existing GitOps workflow — depends on whether feature branches are reconciled by Flux. Existing pattern is direct-to-main commits.) + +- [ ] **Step 5: Verify NodePort live** + +```bash +kubectl --request-timeout=10s -n supervisor get svc ingestion-nodeport 2>&1 +``` + +Expected output includes a service with TYPE=NodePort and PORT(S) `3300:30330/TCP`. + +--- + +## Task 10: End-to-end smoke test against deployed brain MCP + +**Prerequisite:** Hyperguild commits from Tasks 1-8 are pushed to gitea origin and the new ingestion image has rolled out to the cluster (~2 minutes after push, per CD pattern). + +- [ ] **Step 1: Push the hyperguild changes** + +```bash +cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration +git push origin feat/brain-mcp-migration +``` + +- [ ] **Step 2: Open a PR to main, get it reviewed, merge** + +(Manual step. Or fast-forward to main if reviewing solo.) + +- [ ] **Step 3: Wait for image rollout** + +```bash +TARGET=$(git -C /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration rev-parse HEAD) +for i in $(seq 1 30); do + CURRENT=$(kubectl --request-timeout=5s -n supervisor get deploy ingestion -o jsonpath='{.spec.template.spec.containers[0].image}' 2>/dev/null | sed 's/.*://') + if [ "$CURRENT" = "$TARGET" ]; then + echo "[$i] image rolled to $CURRENT" + break + fi + echo "[$i] still on ${CURRENT:0:12} — waiting 10s" + sleep 10 +done +kubectl --request-timeout=8s -n supervisor get pods -l app=ingestion -o wide +``` + +- [ ] **Step 4: Smoke test the live endpoint** + +```bash +echo "=== initialize ===" +curl -sS -X POST http://koala:30330/mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"smoke","version":"0"}}}' +echo "" + +echo "=== tools/list ===" +curl -sS -X POST http://koala:30330/mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | head -c 600 +echo "" + +echo "=== brain_query (existing brain) ===" +curl -sS -X POST http://koala:30330/mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"brain_query","arguments":{"query":"tdd","limit":2}}}' +echo "" + +echo "=== session_log ===" +curl -sS -X POST http://koala:30330/mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"session_log","arguments":{"session_id":"smoke-test","skill":"plan-execution","phase":"verify","final_status":"ok"}}}' +echo "" +``` + +Expected: +- initialize returns `protocolVersion: "2024-11-05"` and `serverInfo.name: "ingestion-brain"`. +- tools/list returns 5 tools. +- brain_query returns whatever's in the brain (or `{"results":null}` if empty). +- session_log returns `{"status":"ok","session_id":"smoke-test"}`. + +If any of these fail: do not proceed to Task 11. Investigate the failure first. + +--- + +## Task 11: Update .mcp.json to use brain MCP + +**Files:** +- Modify: `.mcp.json` (in repo root) + +The supervisor MCP entry stays for now (it still works, and the migration of skill workers is a future plan). We add `brain` as a second MCP server. + +- [ ] **Step 1: Read current `.mcp.json`** + +```bash +cat /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration/.mcp.json +``` + +Expected: + +```json +{ + "mcpServers": { + "supervisor": { + "type": "http", + "url": "http://koala:30320/mcp" + } + } +} +``` + +- [ ] **Step 2: Add brain alongside** + +```json +{ + "mcpServers": { + "supervisor": { + "type": "http", + "url": "http://koala:30320/mcp" + }, + "brain": { + "type": "http", + "url": "http://koala:30330/mcp" + } + } +} +``` + +- [ ] **Step 3: User restarts Claude Code in this project** + +Manual handoff to user: "Restart Claude Code. Then `/mcp` should list two servers: `supervisor` and `brain`. The `brain` server should expose 5 tools (brain_query, brain_write, brain_ingest, brain_ingest_raw, session_log). Try a `brain_query` and confirm it returns results." + +- [ ] **Step 4: Commit only after user confirms** + +```bash +git add .mcp.json +git commit -m "$(cat <<'EOF' +chore: add brain MCP server alongside supervisor + +The brain MCP at koala:30330 hosts the brain_* and session_log tools +formerly on supervisor. Supervisor stays connected during the +transition; its skill workers and the brain duplication will be +removed in a later plan. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 12: Update README and CLAUDE.md + +**Files:** +- Modify: `README.md` +- Modify: `CLAUDE.md` (project-level) + +Document the new MCP endpoint and the dual-MCP transitional state. Keep edits surgical — this is documentation, not a marketing rewrite. + +- [ ] **Step 1: Update README architecture diagram and connect-a-project section** + +Read README.md first. Update the diagram to show two MCP endpoints. Update the "Connect a project" `.mcp.json` example to include both servers. + +```markdown +## Connect a project + +Create `.mcp.json` in your project root: + +```json +{ + "mcpServers": { + "supervisor": { + "type": "http", + "url": "http://koala:30320/mcp" + }, + "brain": { + "type": "http", + "url": "http://koala:30330/mcp" + } + } +} +``` + +Two MCP servers are exposed today, both reachable over Tailscale: + +- **`supervisor`** at `koala:30320` — skill workers (`tdd_*`, `review`, `debug`, `spec`, `retrospective`, `trainer`, `tier`). +- **`brain`** at `koala:30330` — knowledge access (`brain_query`, `brain_write`, `brain_ingest`, `brain_ingest_raw`) and `session_log`. + +The `brain` MCP is hosted by the ingestion service directly. The skill workers will move out of `supervisor` in a later plan. +``` + +- [ ] **Step 2: Add a brief note in CLAUDE.md about the dual MCP** + +Append to the "Knowledge base access" section in `CLAUDE.md`: + +```markdown +- **Brain MCP**: `http://koala:30330/mcp` — preferred path for brain_query, brain_write, brain_ingest_raw, brain_ingest, session_log. +- **Supervisor MCP**: `http://koala:30320/mcp` — skill workers (tdd, review, debug, spec, retrospective, trainer) until they migrate to SKILL.md. +``` + +- [ ] **Step 3: Commit** + +```bash +git add README.md CLAUDE.md +git commit -m "$(cat <<'EOF' +docs: document brain MCP endpoint at koala:30330 + +Updates the connect-a-project example and the CLAUDE.md knowledge +base section. Captures the transitional state where two MCPs coexist; +the supervisor MCP will shrink as skill workers move to SKILL.md. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Task 13: Final verification + +- [ ] **Step 1: Re-run `task check` on the worktree** + +```bash +cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration +task check 2>&1 | tail -20 +``` + +Expected: lint clean, all tests pass, vet clean, govulncheck clean. + +- [ ] **Step 2: Verify both MCPs respond from this Claude Code session** + +After user has restarted Claude Code: +- `/mcp` lists supervisor (connected) and brain (connected). +- Calling `brain_query` via the brain MCP returns results. +- Calling `tier` via the supervisor MCP returns tier info. + +- [ ] **Step 3: Verify `brain/sessions/` has the smoke-test entry** + +```bash +kubectl --request-timeout=8s -n supervisor exec deploy/ingestion -- ls /app/brain/sessions/ 2>&1 | head -5 +``` + +Expected: includes `smoke-test.jsonl` (from Task 10 step 4). + +- [ ] **Step 4: Update plan checkbox status and merge** + +Mark all checkboxes complete in this plan file. Optionally squash the worktree branch into a single feature merge commit before merging to main: + +```bash +cd /Users/mathias/Documents/local-dev/AI/hyperguild +git checkout main +git merge --no-ff feat/brain-mcp-migration -m "feat: brain MCP migration (extract brain + session_log into ingestion pod)" +git push origin main +``` + +- [ ] **Step 5: Clean up the worktree** + +Use `superpowers:finishing-a-development-branch` skill, or manually: + +```bash +git worktree remove /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration +git branch -d feat/brain-mcp-migration +``` + +--- + +## Self-review + +**Spec coverage:** +- ✅ MCP server skeleton (Task 1) +- ✅ session package (Task 2) +- ✅ brain_query (Task 3) +- ✅ brain_write with extracted helper (Task 4) +- ✅ brain_ingest_raw (Task 5) +- ✅ brain_ingest (Task 6) +- ✅ session_log (Task 7) +- ✅ Mount into ingestion server (Task 8) +- ✅ NodePort exposure (Task 9, separate repo) +- ✅ Live cluster verification (Task 10) +- ✅ `.mcp.json` switch (Task 11) +- ✅ Docs (Task 12) +- ✅ Final verification + cleanup (Task 13) + +**Placeholder scan:** No "TBD", "implement later", or hand-wave error handling. Each step has actual code or actual commands. + +**Type consistency:** +- `mcp.NewServer(brainDir string, *pipeline.Config, pipeline.CompleteFunc)` — used consistently across Tasks 1, 3-7, and 8. +- `pipeline.RawPage` shape — referenced in Task 5's test args, matches existing definition. +- `session.Entry` shape — referenced in Task 7, matches Task 2's package definition. +- `api.WriteNote(brainDir, content, filename, type, domain)` — defined Task 4, called from Task 4's MCP handler with positional args matching. + +**Dependencies on out-of-repo work:** +- Task 9 modifies the `~/dev/AI/infra` repo. Pause point built in (Step 4 of Task 9). +- Task 10 depends on CD completing (image build + Flux reconcile, ~2 min). + +**Reversibility:** +- All hyperguild changes are additive until Task 11. The `.mcp.json` change in Task 11 is one line, easily reverted. +- The infra-repo NodePort can be reverted by deleting the new manifest line and re-pushing. +- The supervisor MCP is untouched throughout. If brain MCP fails verification, fall back to supervisor MCP without code changes.