skills/gitea-ci/SKILL.md

---
name: gitea-ci
description: End-to-end Gitea CI/CD setup with self-hosted act_runner, quality gate pipeline, buildah image build, k3s deploy, and GitHub mirror. Covers act_runner-specific gotchas that are not documented anywhere.
---

# Gitea CI/CD Pipeline Setup

## Overview

This skill sets up a complete CI/CD pipeline on Gitea with a self-hosted runner. It covers the full journey from a bare repo to a working pipeline with: lint/test gate, OCI image build, k3s deploy, and GitHub mirror. It also captures non-obvious act_runner quirks that will burn you if you don't know them.

Load this skill when starting a new project that needs CI, or when debugging a broken Gitea Actions pipeline.

## When to Use

- Setting up CI for a new project on this infra (Gitea + koala/iguana/flamingo)
- Debugging a Gitea Actions pipeline that silently fails or behaves unexpectedly
- Migrating an existing project to Gitea Actions from another CI system

---

## Phase 0: Gather Context

Before writing any YAML, collect the following. Use AskUserQuestion for anything not obvious from the project:

| Item | Where to find it | Why it matters |
|------|-----------------|----------------|
| Gitea repo URL | git remote -v | Base URL for API calls |
| Runner machine | CLAUDE.md / infra notes | Which machine runs jobs |
| Primary language | go.mod, package.json, etc. | Drives the toolchain steps |
| Container registry | Ask or check existing config | `localhost:5000` on koala, or external |
| k3s namespace | k8s/ manifests or ask | Deploy target |
| GitHub mirror repo | Ask | Target for the mirror job |
| Image name | Ask or derive from repo name | Used in buildah + k3s steps |

Check the project's `CLAUDE.md` for infra notes first — often this answers most questions without needing to ask.

---

## Phase 1: Verify Runner Is Live

Before writing the workflow, confirm the runner is registered and healthy.

```bash
# Check registered runners via Gitea API
curl -s "https://<gitea-host>/api/v1/repos/<owner>/<repo>/actions/runners" \
  -H "Authorization: token $GITEA_TOKEN" | python3 -m json.tool
```

If no runners appear, the runner is not registered. On this infra, `koala` runs `act_runner`. Check with:
```bash
ssh koala "systemctl status act_runner"
# or if running as a process:
ssh koala "pgrep -a act_runner"
```

act_runner config is typically at `~/.config/act_runner/` or wherever it was installed. Registration command:
```bash
act_runner register \
  --instance https://<gitea-host> \
  --token <runner-registration-token> \
  --name koala \
  --labels self-hosted
```

The registration token is in Gitea: Settings → Actions → Runners → Create Runner.

---

## Phase 2: Write the Workflow

The standard pipeline has 4-5 jobs. Copy the template below and fill in the `env.IMAGE` value and any project-specific toolchain steps.

```yaml
name: CI

on:
  push:
    branches: [main]
    tags: ["v*"]
  pull_request:
    branches: [main]

env:
  IMAGE: <project-name>   # ← change this

jobs:
  # ── 1. Quality gate ─────────────────────────────────────────────────────────
  check:
    name: Lint / Test / Vet
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-go@v5
        with:
          go-version-file: go.mod
          cache: false   # self-hosted: Go cache persists on disk between runs

      - name: Verify toolchain
        run: |
          go version
          task --version
          govulncheck -version 2>&1 || true

      - name: Install golangci-lint
        run: |
          curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/HEAD/install.sh \
            | sh -s -- -b "$(go env GOPATH)/bin" v2.11.4
          golangci-lint --version

      - name: Run checks
        run: task check

  # ── 2. Build image ──────────────────────────────────────────────────────────
  build:
    name: Build & Import
    needs: check
    runs-on: self-hosted
    if: github.event_name != 'pull_request'
    outputs:
      image-tag: ${{ steps.meta.outputs.sha-tag }}
    steps:
      - uses: actions/checkout@v4

      - name: Derive image tags
        id: meta
        run: |
          SHA=$(git rev-parse --short HEAD)
          echo "sha-tag=${SHA}" >> "$GITHUB_OUTPUT"
          REF="${{ github.ref }}"
          if [[ "$REF" == refs/tags/v* ]]; then
            echo "version-tag=${REF#refs/tags/}" >> "$GITHUB_OUTPUT"
          fi

      - name: Build and push to local registry
        run: |
          REGISTRY="localhost:5000"
          REF="${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.sha-tag }}"
          buildah build \
            --label "org.opencontainers.image.revision=${{ github.sha }}" \
            --label "org.opencontainers.image.source=${{ github.repositoryUrl }}" \
            -t ${REF} \
            -t ${REGISTRY}/${{ env.IMAGE }}:latest \
            .
          buildah push --tls-verify=false ${REF}
          buildah push --tls-verify=false ${REGISTRY}/${{ env.IMAGE }}:latest
          [[ -n "${{ steps.meta.outputs.version-tag }}" ]] && \
            buildah push --tls-verify=false \
              ${REF} \
              ${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.version-tag }} || true
          echo "✓ Image pushed to ${REF}"

      - name: Smoke test
        run: |
          REGISTRY="localhost:5000"
          REF="${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.sha-tag }}"
          CNAME="smoke-${{ steps.meta.outputs.sha-tag }}"
          sudo k3s ctr images pull --plain-http ${REF}
          OUTPUT=$(timeout 5 sudo k3s ctr run --rm ${REF} ${CNAME} /<binary-name> 2>&1 || true)
          sudo k3s ctr containers delete ${CNAME} 2>/dev/null || true
          echo "$OUTPUT" | grep -q "<project-name>" \
            && echo "✓ Smoke test passed" \
            || echo "⚠ Smoke test inconclusive: $OUTPUT"

  # ── 3. Deploy to k3s ────────────────────────────────────────────────────────
  deploy:
    name: Deploy to staging (k3s)
    needs: build
    runs-on: self-hosted
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    environment: staging
    steps:
      - uses: actions/checkout@v4

      - name: Apply manifests and update image
        env:
          IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
        run: |
          kubectl apply -f k8s/namespace.yaml
          kubectl apply -f k8s/deployment.yaml
          kubectl set image deployment/<project-name> \
            <project-name>=localhost:5000/${{ env.IMAGE }}:${IMAGE_TAG} \
            --namespace <project-name>

      - name: Verify rollout
        run: |
          kubectl rollout status deployment/<project-name> \
            --namespace <project-name> \
            --timeout=120s \
          || {
            echo "── pod status ──"
            kubectl get pods -n <project-name> -o wide
            echo "── pod events ──"
            kubectl get events -n <project-name> --sort-by='.lastTimestamp' | tail -20
            exit 1
          }

  # ── 4. Mirror to GitHub ─────────────────────────────────────────────────────
  mirror:
    name: Mirror to GitHub
    needs: deploy
    runs-on: self-hosted
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Push to GitHub
        run: |
          mkdir -p ~/.ssh
          echo '${{ secrets.GH_DEPLOY_KEY }}' > ~/.ssh/id_rsa_gh_mirror
          chmod 600 ~/.ssh/id_rsa_gh_mirror
          ssh-keyscan github.com >> ~/.ssh/known_hosts 2>/dev/null
          GIT_SSH_COMMAND="ssh -i ~/.ssh/id_rsa_gh_mirror -o IdentitiesOnly=yes" \
            git push git@github.com:<gh-org>/<repo>.git HEAD:main
          rm ~/.ssh/id_rsa_gh_mirror
          echo "✓ Mirrored to GitHub"
```

Replace all `<placeholders>` before committing.

---

## Trunk-Based Development and CI

This infra uses TBD. CI on every push to main is the quality gate — not branch protection.

### What this means in practice

- Commit directly to main for all solo and single-agent work
- Each commit: one logical change + `task check` passing locally before push
- CI runs `task check` again on push — if it was green locally it will be green in CI
- No PRs, no feature branches, no squash-merges for normal work

### The pre-push ritual

```bash
# In project root before every push:
task check && git push origin main
```

Wire this into your shell or a git pre-push hook so it's automatic.

### When CI catches something you missed locally

Fix forward — commit the fix directly to main. Do not revert unless the broken
commit introduced a data migration or irreversible infrastructure change.

### Feature flags instead of feature branches

If a feature is too large for one session, hide it behind a build tag or a config
flag until ready:

```go
// cmd/server/main.go
if os.Getenv("ENABLE_EXPERIMENTAL_FEATURE") == "true" {
    server.RegisterExperimentalRoutes(mux)
}
```

Commit the hidden feature incrementally. Activate it when complete.

### When to break the rule

- **Parallel agents on the same repo:** short-lived `agent/<description>` branch,
  merge to main inside the same session.
- **External contributor or client four-eyes requirement:** PR flow, document the
  reason in `PROJECT.md`.

---

## Phase 3: Set Up GitHub Mirror

The mirror job needs an SSH deploy key with write access to the GitHub repo.

```bash
# On your local machine:
ssh-keygen -t ed25519 -C "gitea-mirror-<project>" -f /tmp/gh_mirror_key -N ""
cat /tmp/gh_mirror_key.pub   # add this to GitHub repo → Settings → Deploy Keys (Allow write)
cat /tmp/gh_mirror_key       # add this to Gitea repo → Settings → Secrets → GH_DEPLOY_KEY
rm /tmp/gh_mirror_key /tmp/gh_mirror_key.pub
```

The private key goes in Gitea as the `GH_DEPLOY_KEY` secret. The public key goes in GitHub as a deploy key with write access.

---

## Phase 4: Add Project Secrets

Set secrets via the Gitea API — faster than the UI and scriptable:

```bash
# Template:
curl -s -X PUT \
  "https://<gitea-host>/api/v1/repos/<owner>/<repo>/actions/secrets/<SECRET_NAME>" \
  -H "Authorization: token $GITEA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"data":"<secret-value>"}' \
  -w "\nHTTP %{http_code}"
# Expected: HTTP 204

# Verify secrets are set:
curl -s "https://<gitea-host>/api/v1/repos/<owner>/<repo>/actions/secrets" \
  -H "Authorization: token $GITEA_TOKEN" | python3 -m json.tool
```

Required secrets for the baseline pipeline: `GH_DEPLOY_KEY`.

Any project-specific secrets (API keys, tokens, credentials) are added the same way.

---

## Phase 5: Debug Failing Runs

When a run fails, use the Gitea API to get logs without opening a browser.

```bash
GITEA_HOST="https://<gitea-host>"
REPO="<owner>/<repo>"
TOKEN="$GITEA_TOKEN"

# List recent runs
curl -s "$GITEA_HOST/api/v1/repos/$REPO/actions/runs?limit=5" \
  -H "Authorization: token $TOKEN" | python3 -c "
import json,sys
for r in json.load(sys.stdin)['workflow_runs']:
    print(f'#{r[\"id\"]} {r[\"status\"]:12} {r.get(\"conclusion\",\"\"):10} {r[\"display_title\"][:50]}')
"

# List jobs for a run (replace RUN_ID)
curl -s "$GITEA_HOST/api/v1/repos/$REPO/actions/runs/<RUN_ID>/jobs" \
  -H "Authorization: token $TOKEN" | python3 -c "
import json,sys
for j in json.load(sys.stdin)['jobs']:
    print(f'Job {j[\"id\"]}: {j[\"name\"]:30} | {j[\"status\"]:10} | {j.get(\"conclusion\",\"\")}')
"

# Get full log for a job (replace JOB_ID)
curl -s "$GITEA_HOST/api/v1/repos/$REPO/actions/jobs/<JOB_ID>/logs" \
  -H "Authorization: token $TOKEN" | grep -E "ERROR|error|FAIL|undefined|cannot" | tail -30
```

---

## ⚠ act_runner Gotchas

These are non-obvious bugs and sharp edges that will silently break your pipeline. Read all of them.

### 1. Steps with `env:` blocks that use secret expressions are silently skipped

**Symptom**: A step produces no log output at all — not even its step header (`⭐ Run Main ...`). Subsequent steps fail as if the skipped step never ran.

**Cause**: In act_runner v0.4.x, steps that have an `env:` block containing `${{ secrets.* }}` expressions are silently dropped before execution.

**Wrong**:
```yaml
- name: Write config
  env:
    API_KEY: ${{ secrets.API_KEY }}
  run: |
    echo "KEY=$API_KEY" > config.env
```

**Right** — inline the expression directly in the `run:` script:
```yaml
- name: Write config
  run: |
    echo 'KEY=${{ secrets.API_KEY }}' > config.env
```

act_runner evaluates `${{ }}` expressions before passing the script to bash. The values are redacted in log output. No `env:` block needed.

### 2. Heredocs in `run: |` blocks — the indented EOF trap

**Symptom**: A file is written with 10-100× more content than expected. JSON files fail to parse. The step appears to succeed.

**Cause**: In a YAML literal block scalar (`run: |`), all content is at the block's indentation level. If you indent your heredoc content (to look neat), the `EOF` terminator also gets indented. Bash requires the terminator to be at column 0 — an indented `EOF` is NOT recognized, so the heredoc consumes everything that follows until end-of-script.

**Wrong**:
```yaml
      - name: Write file
        run: |
          cat > output.json <<EOF
          {
            "key": "value"
          }
          EOF          ← indented, NOT recognized as terminator
          chmod 600 output.json
```

The file ends up containing the JSON, the literal string `          EOF`, and `          chmod 600 output.json`.

**Right** — avoid heredocs entirely. Use `echo` for simple lines, `printf` for single-line formatted output:
```yaml
      - name: Write file
        run: |
          echo '{"key":"${{ secrets.SOME_VALUE }}"}' > output.json
          chmod 600 output.json
```

For JSON with dynamic values, use python3 as a one-liner (but see gotcha #3):
```yaml
      - name: Write token file
        run: |
          python3 -c "import json; open('token.json','w').write(json.dumps({'key':'${{ secrets.TOKEN }}','type':'bearer'}))"
```

### 3. Multiline `python3 -c` breaks YAML block scalar parsing

**Symptom**: The workflow file is pushed but no CI run appears — Gitea silently rejects the workflow YAML.

**Cause**: Unindented code in a multiline `python3 -c "..."` block (with line breaks inside the YAML `run: |`) is at column 0, which the YAML parser interprets as ending the block scalar and starting a new YAML key.

**Wrong**:
```yaml
        run: |
          python3 -c "
import json          ← column 0, YAML parser thinks block scalar ended
d = {...}
"
```

**Right** — keep the python3 invocation on one line:
```yaml
        run: |
          python3 -c "import json; d = {...}; open('f','w').write(json.dumps(d))"
```

If the one-liner is too long, write a small `.py` file in a previous step and execute it.

### 4. Validate YAML locally before pushing

Gitea silently ignores workflows with YAML parse errors — no run appears, no error message.

```bash
python3 -c "import yaml; yaml.safe_load(open('.gitea/workflows/ci.yml').read()); print('YAML valid')"
```

Run this before every push that touches the workflow file.

### 5. Commit generated files instead of gitignoring them

**Symptom**: Lint/build fails in CI with `undefined: SomeGeneratedType` even though it works locally.

**Cause**: Code generation tools (templ, sqlc, wire, protoc, etc.) produce `_generated.go` or `_templ.go` files that are often in `.gitignore`. CI runners don't have the generator installed and can't regenerate them.

**Fix**: Remove the generated files from `.gitignore` and commit them. They are deterministic, diffable Go source — committing them is the standard practice for projects that don't want to install generators in CI.

If you need generators in CI anyway (e.g., for proto changes), add a generation step BEFORE lint in the `check` job and install the tool explicitly.

### 6. `errcheck` will flag `fmt.Sscanf` and similar "can't fail in practice" calls

Parsers that convert strings to numbers (e.g., parsing API responses where you know the format) will trigger `errcheck`. Suppress with blank assignment:

```go
// before
fmt.Sscanf(s, "%d", &n)

// after — communicates intent and satisfies errcheck
_, _ = fmt.Sscanf(s, "%d", &n)
```

---

## Verification Checklist

Before marking CI setup complete:

- [ ] `python3 -c "import yaml; yaml.safe_load(...)"` passes on the workflow YAML
- [ ] Runner appears as active in Gitea repo → Settings → Actions → Runners
- [ ] A push to `main` triggers a run visible in Gitea repo → Actions
- [ ] `check` job passes (lint + test + vet green)
- [ ] `build` job produces an image in the local registry (`curl localhost:5000/v2/<image>/tags/list`)
- [ ] `deploy` job shows rollout success in pod logs
- [ ] `mirror` job shows "✓ Mirrored to GitHub" and the commit appears on GitHub
- [ ] A PR (non-main push) runs `check` only, skips build/deploy/mirror
- [ ] Any project-specific secrets are set via API and verified with a secrets list call

---

## Related Skills

- `tdd` — for writing the tests that the `check` job runs
- `planning` — for breaking down the work before writing pipeline YAML
- `problem-analysis` — for debugging when a job fails in unexpected ways