> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ionworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# 2026 07 20 protocols require project id pr1 expand

# Protocols require project\_id — PR 1 (expand) Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add nullable `project_id` scoping to protocols (`experiment_templates`) and their parameterisations (`experiments`), seed every project with copies of the built-in protocol catalog, and make reads/writes project-aware — all backward-compatible (expand phase).

**Architecture:** Follow the `optimization_templates`/`jobs` `access_level` precedent. The existing system-org rows (`organization_id = 00000000-0000-0000-0000-000000000001`) become a hidden `access_level='system'` catalog. New projects get project-scoped copies synchronously on creation; existing projects are backfilled by an `on_demand` ops task. The canonical-dedup RPC and the experiments upsert arbiter are made project-aware (both would otherwise be correctness bugs). Reads return a project's own rows when a `project_id` is in scope; the legacy org+system union stays for callers that don't yet pass one.

**Tech Stack:** FastAPI, Supabase (PostgREST + Postgres 15), Pydantic, pytest. `just` for tests, `uv` for Python. Spec: `docs/superpowers/specs/2026-07-20-protocols-require-project-id-design.md`.

**Scope:** This plan is **PR 1 only**. PR 2 (contract: VALIDATE FKs, `SET NOT NULL`, delete catalog, remove system-org union) and PR 3 (project-membership RLS) are separate, time-gated efforts and are NOT planned here.

**Conventions to honour (from repo rules):**

* Migrations are DDL-only + idempotent (`.claude/rules/supabase-migrations.md`). Data changes → ops task.
* Services raise domain exceptions, never `HTTPException`/`ValueError` (`.claude/rules/fastapi-backend.md`).
* Repos paginate; list methods keep `count`/`total` shape (`.claude/rules/repositories.md`).
* No unit tests for ops tasks (user preference).
* Run tests via `just`, never bare pytest. Use `uv` for Python.

***

## File Structure

**Create:**

* `supabase/migrations/20260728174630_add_project_id_to_protocols.sql` — expand DDL (columns, FKs, indexes, project-aware `find_by_content` RPC).
* `backend/src/ops_tasks/0022_seed_project_protocols.py` — backfill: stamp catalog, copy into every existing project.
* `backend/src/services/protocol_catalog.py` — shared `copy_catalog_protocols_into_project` helper (used by both project creation and the ops task).
* `backend/tests/test_services/test_protocol_catalog.py` — helper unit tests.

**Modify:**

* `backend/src/pydantic_models/experiments.py` — add `project_id` / `access_level` to `ExperimentTemplate`; `project_id` to `Experiment`.
* `backend/src/repositories/experiment_templates.py` — `LIST_DEFAULT_COLUMNS`, project\_id filter on listing, project-aware `find_or_create_canonical_template`.
* `backend/src/repositories/experiments.py` — 4-column `on_conflict`, project\_id in payload + finder.
* `backend/src/services/project_service.py` — seed catalog on `create_project_with_admin`.
* `backend/src/routes/experiment_templates.py` — `project_id` query param on list; require `project_id` on create; stamp `access_level`.
* `backend/tests/test_services/test_project_service.py` — seed-on-create + idempotency tests.
* `backend/tests/test_repositories/test_experiment_templates_conflict.py` — project-aware RPC threading test.

***

## Task 1: Add `project_id` / `access_level` to the Pydantic models

**Files:**

* Modify: `backend/src/pydantic_models/experiments.py`

* [ ] **Step 1: Add fields to `ExperimentTemplate`**

After the `created_by_email` field in `class ExperimentTemplate`, add:

```python theme={null}
    project_id: str | None = Field(
        default=None,
        description="ID of the project that owns this protocol (None for the system catalog)",
    )
    access_level: str = Field(
        default="project",
        description="'system' for catalog rows, 'project' for project-owned protocols",
    )
```

* [ ] **Step 2: Add `project_id` to `Experiment`**

In `class Experiment`, after `organization_id`, add:

```python theme={null}
    project_id: str | None = Field(
        default=None,
        description="ID of the project that owns this experiment (inherited from its template)",
    )
```

* [ ] **Step 3: Verify import compiles**

Run: `cd backend && uv run python -c "from src.pydantic_models.experiments import ExperimentTemplate, Experiment; print('ok')"`
Expected: prints `ok`.

* [ ] **Step 4: Commit**

```bash theme={null}
git add backend/src/pydantic_models/experiments.py
git commit -m "feat(protocols): add project_id/access_level to experiment models"
```

***

## Task 2: Expand migration (columns, FKs, indexes, project-aware RPC)

**Files:**

* Create: `supabase/migrations/20260728174630_add_project_id_to_protocols.sql`

> **DDL-only + idempotent.** The `access_level='system'` data stamp lives in the ops task (Task 6), NOT here.

* [ ] **Step 1: Write the migration**

```sql theme={null}
-- Expand phase: add nullable project_id + access_level to experiment_templates
-- and nullable project_id to experiments, scoping protocols to a project.
-- Mirrors the optimization_templates / jobs access_level pattern. A follow-up
-- contract migration (separate PR, ≥1 month later) validates the FKs and sets
-- project_id NOT NULL once ops task 0022 has run in every environment.

-- ── experiment_templates ────────────────────────────────────────────────────
ALTER TABLE public.experiment_templates
    ADD COLUMN IF NOT EXISTS project_id uuid;

-- Default 'system' (NOT 'project') is load-bearing: the only pre-existing rows
-- are the system-org catalog with project_id NULL, which satisfy only the
-- 'system' arm of the invariant CHECK below. Defaulting to 'project' would make
-- the CHECK fail on add. New project-scoped writes set 'project' explicitly.
ALTER TABLE public.experiment_templates
    ADD COLUMN IF NOT EXISTS access_level text NOT NULL DEFAULT 'system';

DO $$ BEGIN
    IF NOT EXISTS (
        SELECT 1 FROM pg_constraint
        WHERE conname = 'experiment_templates_access_level_check'
    ) THEN
        ALTER TABLE public.experiment_templates
            ADD CONSTRAINT experiment_templates_access_level_check
            CHECK (access_level IN ('system', 'project'));
    END IF;
END $$;

DO $$ BEGIN
    IF NOT EXISTS (
        SELECT 1 FROM pg_constraint
        WHERE conname = 'experiment_templates_access_level_project_id_check'
    ) THEN
        ALTER TABLE public.experiment_templates
            ADD CONSTRAINT experiment_templates_access_level_project_id_check
            CHECK (
                (access_level = 'project' AND project_id IS NOT NULL)
                OR (access_level = 'system' AND project_id IS NULL)
            );
    END IF;
END $$;

DO $$ BEGIN
    IF NOT EXISTS (
        SELECT 1 FROM pg_constraint
        WHERE conname = 'experiment_templates_project_id_organization_id_fkey'
    ) THEN
        ALTER TABLE public.experiment_templates
            ADD CONSTRAINT experiment_templates_project_id_organization_id_fkey
            FOREIGN KEY (project_id, organization_id)
            REFERENCES public.projects(id, organization_id)
            ON DELETE CASCADE
            NOT VALID;
    END IF;
END $$;

CREATE INDEX IF NOT EXISTS idx_experiment_templates_project_id
    ON public.experiment_templates (project_id);

-- Project-aware dedup: each project holds its own copy; a single system slot
-- (project_id NULL) stays deduped via NULLS NOT DISTINCT, and the fixed column
-- list stays targetable by PostgREST on_conflict.
DROP INDEX IF EXISTS public.idx_experiment_templates_content_hash_org;
CREATE UNIQUE INDEX IF NOT EXISTS idx_experiment_templates_content_hash_org_proj
    ON public.experiment_templates (content_hash, organization_id, project_id)
    NULLS NOT DISTINCT;

-- ── experiments ──────────────────────────────────────────────────────────────
ALTER TABLE public.experiments
    ADD COLUMN IF NOT EXISTS project_id uuid;

DO $$ BEGIN
    IF NOT EXISTS (
        SELECT 1 FROM pg_constraint
        WHERE conname = 'experiments_project_id_organization_id_fkey'
    ) THEN
        ALTER TABLE public.experiments
            ADD CONSTRAINT experiments_project_id_organization_id_fkey
            FOREIGN KEY (project_id, organization_id)
            REFERENCES public.projects(id, organization_id)
            ON DELETE CASCADE
            NOT VALID;
    END IF;
END $$;

CREATE INDEX IF NOT EXISTS idx_experiments_project_id
    ON public.experiments (project_id);

DROP INDEX IF EXISTS public.experiments_template_params_org_unique;
CREATE UNIQUE INDEX IF NOT EXISTS experiments_template_params_org_proj_unique
    ON public.experiments (template_id, parameters, organization_id, project_id)
    NULLS NOT DISTINCT;

-- ── project-aware canonical find ──────────────────────────────────────────────
-- The old signature keyed on (content_hash, organization_id) only, so after the
-- backfill (identical content_hash across project copies) it returned an
-- arbitrary project's row. Add in_project_id; IS NOT DISTINCT FROM matches the
-- NULL catalog slot when passed NULL and only the exact project row otherwise.
CREATE OR REPLACE FUNCTION public.experiment_templates_find_by_content(
    in_protocol_config jsonb,
    in_parameters_schema jsonb,
    in_organization_id uuid,
    in_project_id uuid
) RETURNS SETOF public.experiment_templates
    LANGUAGE sql STABLE
    AS $$
  SELECT *
  FROM public.experiment_templates et
  WHERE et.content_hash = public.generate_experiment_template_content_hash(
          in_protocol_config, in_parameters_schema)
    AND et.organization_id = in_organization_id
    AND et.project_id IS NOT DISTINCT FROM in_project_id
  LIMIT 1;
$$;
```

> Note: this `CREATE OR REPLACE` changes the function signature (adds a param). Because the arg list differs, Postgres creates a **new overload** rather than replacing the 3-arg one. Drop the old overload explicitly so only the 4-arg version remains:

```sql theme={null}
DROP FUNCTION IF EXISTS public.experiment_templates_find_by_content(jsonb, jsonb, uuid);
```

Place the `DROP FUNCTION` **before** the `CREATE OR REPLACE`.

> **Re-grant EXECUTE (required).** `DROP FUNCTION` removes the 3-arg overload's
> grants (`baseline.sql:6393-6395`), and because the new function has a different
> arg list `CREATE OR REPLACE` does not re-issue them. The RPC is called via the
> **user** client (`experiment_templates.py:167,220`), so `authenticated` needs
> EXECUTE — Postgres grants EXECUTE to PUBLIC by default, but a hardened Supabase
> that revoked PUBLIC would break the call. Add, after the `CREATE OR REPLACE`:

```sql theme={null}
GRANT EXECUTE ON FUNCTION public.experiment_templates_find_by_content(jsonb, jsonb, uuid, uuid)
    TO anon, authenticated, service_role;
```

* [ ] **Step 2: Apply migration against local Supabase**

Run: `supabase migration up` (from repo root; local stack already running per session setup).
Expected: applies with no error; existing system-org rows now have `access_level='system'`, `project_id NULL` (satisfies the CHECK).

* [ ] **Step 3: Verify the CHECK holds and RPC signature swapped**

Run:

```bash theme={null}
psql "$SUPABASE_DB_URL" -c "\d public.experiment_templates" | grep -E "access_level|project_id"
psql "$SUPABASE_DB_URL" -c "\df public.experiment_templates_find_by_content"
```

Expected: columns present; exactly one `find_by_content` overload with 4 args.

(If `$SUPABASE_DB_URL` is unset, use the worktree's `postgresql://postgres:postgres@127.0.0.1:54322/postgres`.)

* [ ] **Step 4: Commit**

```bash theme={null}
git add supabase/migrations/20260728174630_add_project_id_to_protocols.sql
git commit -m "feat(protocols): expand migration — project_id, access_level, project-aware dedup + RPC"
```

***

## Task 3: Project-aware canonical dedup in the templates repo

**Files:**

* Modify: `backend/src/repositories/experiment_templates.py`

* Test: `backend/tests/test_repositories/test_experiment_templates_conflict.py`

* [ ] **Step 1: Write the failing test (RPC receives project\_id)**

Add to `test_experiment_templates_conflict.py`:

```python theme={null}
@pytest.mark.asyncio
async def test_find_or_create_threads_project_id_into_rpc():
    """project_id is passed to the RPC and into the create payload."""
    repo = _make_repo()
    _wire_rpc(repo, [MagicMock(data=[])])  # no existing → create path
    repo.model_class = MagicMock(side_effect=lambda **kw: MagicMock(**kw))
    created = MagicMock(id="tmpl-new")
    repo.create = AsyncMock(return_value=created)

    await repo.find_or_create_canonical_template(
        name="P",
        protocol_config={"steps": []},
        parameters_schema={},
        organization_id="org-1",
        project_id="proj-1",
    )

    rpc_kwargs = repo.supabase.rpc.call_args.args[1]
    assert rpc_kwargs["in_project_id"] == "proj-1"
    create_payload = repo.create.call_args.args[0]
    assert create_payload["project_id"] == "proj-1"
    assert create_payload["access_level"] == "project"
```

* [ ] **Step 2: Run test to verify it fails**

Run: `just test-backend tests/test_repositories/test_experiment_templates_conflict.py::test_find_or_create_threads_project_id_into_rpc -v`
Expected: FAIL — `find_or_create_canonical_template` has no `project_id` param.

* [ ] **Step 3: Add `project_id` param and thread it through**

In `find_or_create_canonical_template` (`experiment_templates.py:148`):

* Add `project_id: str | None = None,` to the signature (after `organization_id`).

* Add `"in_project_id": project_id,` to `rpc_params`.

* In the `template_data` dict for the create path, add:
  ```python theme={null}
  "project_id": project_id,
  "access_level": "project" if project_id is not None else "system",
  ```

* [ ] **Step 4: Run test to verify it passes**

Run: `just test-backend tests/test_repositories/test_experiment_templates_conflict.py -v`
Expected: PASS (including the existing race test — confirm `rpc_params` reuse still works).

* [ ] **Step 5: Commit**

```bash theme={null}
git add backend/src/repositories/experiment_templates.py backend/tests/test_repositories/test_experiment_templates_conflict.py
git commit -m "feat(protocols): thread project_id through canonical find-or-create"
```

***

## Task 4: Project-aware listing + LIST\_DEFAULT\_COLUMNS

**Files:**

* Modify: `backend/src/repositories/experiment_templates.py`

* [ ] **Step 1: Add new columns to `LIST_DEFAULT_COLUMNS`**

Change `LIST_DEFAULT_COLUMNS` (`experiment_templates.py:16-20`) to include the two new columns:

```python theme={null}
LIST_DEFAULT_COLUMNS = (
    "id, name, description, description_template, organization_id,"
    " project_id, access_level, created_at, updated_at, created_by"
)
```

* [ ] **Step 2: Add a project-scoped list method**

Add a method that lists a single project's protocols. Mirror the **exact**
projection idiom from `get_templates_by_organization_with_email`
(`experiment_templates.py:124-128`) — there is **no** `_build_select_columns`
helper; the projection is built inline. Copy that block verbatim and add the
project/access filters:

```python theme={null}
    async def get_templates_by_project_with_email(
        self,
        organization_id: str,
        project_id: str,
        *,
        limit: int | None = None,
        offset: int | None = None,
        include: set[str] | None = None,
    ) -> tuple[list[ExperimentTemplate], int]:
        """List a project's protocols (access_level='project'), with creator email."""
        extra = sorted((include or set()) & INCLUDABLE_COLUMNS)
        select_cols = LIST_DEFAULT_COLUMNS
        if extra:
            select_cols = f"{select_cols}, {', '.join(extra)}"
        select_cols = f"{select_cols}, users(email)"

        query = (
            self.supabase.table(self.table_name)
            .select(select_cols, count="exact")
            .eq("organization_id", organization_id)
            .eq("project_id", project_id)
            .eq("access_level", "project")
            .order("created_at", desc=True)
        )
        if limit is not None and offset is not None:
            query = query.range(offset, offset + limit - 1)
        elif limit is not None:
            query = query.range(0, limit - 1)

        response = await query.execute()
        templates = [
            self.model_class(**self._extract_created_by_email(row))
            for row in (response.data or [])
        ]
        return templates, response.count or 0
```

Note the row mapping uses the existing `self._extract_created_by_email(row)` (as
the org method does), not `self.model_class(**row)` directly — the `users(email)`
join needs flattening.

* [ ] **Step 3: Verify it imports/compiles**

Run: `cd backend && uv run python -c "from src.repositories.experiment_templates import ExperimentTemplateRepository; print('ok')"`
Expected: `ok`.

* [ ] **Step 4: Commit**

```bash theme={null}
git add backend/src/repositories/experiment_templates.py
git commit -m "feat(protocols): project-scoped template listing + expose project_id/access_level"
```

***

## Task 5: Shared catalog-copy helper

**Files:**

* Create: `backend/src/services/protocol_catalog.py`

* Test: `backend/tests/test_services/test_protocol_catalog.py`

* [ ] **Step 0: GATING — validate `on_conflict` on a trigger-set column (do this first)**

The whole seed path (this helper + Task 6 ops task + Task 7 create) hinges on
whether PostgREST accepts an `upsert` whose `on_conflict` arbiter names
`content_hash` — a column set by the BEFORE-INSERT trigger and **absent from the
insert payload**. No existing code in this repo does this, so validate it once,
locally, before writing the helper, and **record the decision in this step**:

Run a throwaway script against the local stack (a project must already have a
`project_id`; the catalog rows must exist):

```bash theme={null}
cd backend && uv run python - <<'PY'
import asyncio
from src.deps import get_supabase_async_service_client  # or however the service client is built in a script
# insert one catalog copy for a real (org, project) via
#   .upsert([{...no content_hash...}], on_conflict="content_hash,organization_id,project_id", ignore_duplicates=True)
# then run it a SECOND time and assert the row count did not grow.
PY
```

* **If the upsert succeeds and the replay is a no-op** → use the `upsert` form in
  Step 3 (and Task 6). Record: "Decision: on\_conflict upsert works."
* **If PostgREST rejects the arbiter** → use the per-row fallback: for each
  catalog row, call the project-aware `experiment_templates_find_by_content` RPC
  (with `in_project_id`); if it returns a row, skip; else insert. This is the
  same idempotency guarantee, just explicit. Record: "Decision: per-row
  existence-check fallback." Then write Step 3 and Task 6 in that form.

Do not proceed to Step 1 until this is decided and recorded.

* [ ] **Step 1: Write the failing test**

```python theme={null}
"""Tests for copy_catalog_protocols_into_project."""

from unittest.mock import AsyncMock, MagicMock

import pytest
from src.services.protocol_catalog import copy_catalog_protocols_into_project

SYSTEM_ORG = "00000000-0000-0000-0000-000000000001"


def _catalog_row(name: str) -> dict:
    return {
        "id": f"cat-{name}",
        "name": name,
        "protocol_config": {"steps": []},
        "parameters_schema": {},
        "time_series_spec": {},
        "metrics_spec": {},
        "plot_options": {},
        "description": None,
        "source_protocol": None,
    }


@pytest.mark.asyncio
async def test_copies_each_catalog_row_scoped_to_project():
    service = MagicMock()
    # catalog read returns two system rows
    read = AsyncMock(return_value=MagicMock(data=[_catalog_row("Charge"), _catalog_row("Discharge")]))
    service.table.return_value.select.return_value.eq.return_value.eq.return_value.execute = read
    insert_exec = AsyncMock(return_value=MagicMock(data=[{"id": "new"}]))
    service.table.return_value.upsert.return_value.execute = insert_exec

    n = await copy_catalog_protocols_into_project(service, "org-1", "proj-1")

    assert n == 2
    # each inserted row is project-scoped
    payloads = [c.args[0] for c in service.table.return_value.upsert.call_args_list]
    flat = [row for p in payloads for row in (p if isinstance(p, list) else [p])]
    assert all(r["project_id"] == "proj-1" for r in flat)
    assert all(r["organization_id"] == "org-1" for r in flat)
    assert all(r["access_level"] == "project" for r in flat)
```

> Adjust the mock-chaining to match the actual PostgREST fluent calls you write in Step 3 (read = `.select(...).eq("access_level","system").eq(...)`; write = `.upsert(..., on_conflict=..., ignore_duplicates=True)`). Keep the assertions.

* [ ] **Step 2: Run test to verify it fails**

Run: `just test-backend tests/test_services/test_protocol_catalog.py -v`
Expected: FAIL — module does not exist.

* [ ] **Step 3: Implement the helper**

```python theme={null}
"""Copy the built-in protocol catalog into a project.

The catalog is the set of system experiment_templates
(``access_level='system'``, ``organization_id`` = the system org). Each project
gets its own editable copies on creation and via the backfill ops task. The copy
is keyed so replays are idempotent against the project-aware dedup index.
"""

import logging

from supabase import AsyncClient

logger = logging.getLogger(__name__)

SYSTEM_ORGANIZATION_ID = "00000000-0000-0000-0000-000000000001"

_COPY_FIELDS = (
    "name",
    "protocol_config",
    "parameters_schema",
    "time_series_spec",
    "metrics_spec",
    "plot_options",
    "description",
    "source_protocol",
)


async def copy_catalog_protocols_into_project(
    service_client: AsyncClient,
    organization_id: str,
    project_id: str,
    created_by: str | None = None,
) -> int:
    """Copy every catalog protocol into ``project_id`` under ``organization_id``.

    Uses the **service** client because catalog rows live in the system org and a
    user client cannot read them. Idempotent: an ``upsert`` with
    ``ignore_duplicates`` against the project-aware unique index makes a replay a
    no-op (a same-content row already present for this (org, project) is skipped).

    Parameters
    ----------
    service_client : AsyncClient
        Supabase service-role client.
    organization_id : str
        Org that will own the copies (the project's org).
    project_id : str
        Destination project.
    created_by : str | None, optional
        User id to stamp on the copies. Defaults to None.

    Returns
    -------
    int
        Number of catalog rows processed (copied or already present).
    """
    catalog = (
        await service_client.table("experiment_templates")
        .select(", ".join(("id",) + _COPY_FIELDS))
        .eq("organization_id", SYSTEM_ORGANIZATION_ID)
        .eq("access_level", "system")
        .execute()
    )
    rows = catalog.data or []
    if not rows:
        return 0

    payload = [
        {
            **{f: row.get(f) for f in _COPY_FIELDS},
            "organization_id": organization_id,
            "project_id": project_id,
            "access_level": "project",
            "created_by": created_by,
        }
        for row in rows
    ]
    # content_hash is recomputed by the BEFORE INSERT trigger; the project-aware
    # unique index (content_hash, organization_id, project_id) NULLS NOT DISTINCT
    # makes ignore_duplicates a safe replay no-op.
    await (
        service_client.table("experiment_templates")
        .upsert(
            payload,
            on_conflict="content_hash,organization_id,project_id",
            ignore_duplicates=True,
        )
        .execute()
    )
    return len(rows)
```

> **Verify before relying on it:** confirm `on_conflict` can target the `content_hash` column even though the app never sets it (the trigger does). If PostgREST rejects an `on_conflict` arbiter on a column absent from the insert payload, fall back to a per-row existence check (`find_by_content` RPC with `in_project_id`) instead of `upsert`. Decide this during implementation with a quick local test; keep the helper's return contract identical.

* [ ] **Step 4: Run test to verify it passes**

Run: `just test-backend tests/test_services/test_protocol_catalog.py -v`
Expected: PASS.

* [ ] **Step 5: Commit**

```bash theme={null}
git add backend/src/services/protocol_catalog.py backend/tests/test_services/test_protocol_catalog.py
git commit -m "feat(protocols): shared catalog-copy helper for project seeding"
```

***

## Task 6: Backfill ops task

**Files:**

* Create: `backend/src/ops_tasks/0022_seed_project_protocols.py`

> No unit test (ops-task preference). Verify by running locally.

* [ ] **Step 1: Write the ops task**

```python theme={null}
"""Seed every existing project with copies of the built-in protocol catalog.

Insert-only (no claim/mutate/delete): the system-org catalog stays intact as the
source of truth through the expand phase. Idempotent — re-running finds every
catalog protocol already present per project and inserts nothing. on_demand so
it is run and reviewed per environment.

Follow-up (separate PR, once run everywhere): the contract migration makes
experiment_templates/experiments project_id NOT NULL and removes the system-org
catalog + union code path.
"""

import asyncio
import logging

from supabase import Client

from src.services.protocol_catalog import (
    SYSTEM_ORGANIZATION_ID,
    copy_catalog_protocols_into_project,
)

RUN_ON = "on_demand"

logger = logging.getLogger(__name__)

_BATCH = 500


def run(supabase: Client) -> None:
    """Stamp the catalog then copy it into every project (see module docstring)."""
    # 1. Defensive re-stamp: any system-org row that isn't labelled 'system'
    #    (should be none after the migration default) is corrected.
    supabase.table("experiment_templates").update({"access_level": "system"}).eq(
        "organization_id", SYSTEM_ORGANIZATION_ID
    ).neq("access_level", "system").execute()

    # 2. Copy the catalog into every project, keyset-paginated by id.
    #    copy_catalog_protocols_into_project is async (built for the AsyncClient
    #    request path); run it on the sync ops client via a fresh event loop.
    projects: list[dict] = []
    cursor: str | None = None
    while True:
        q = (
            supabase.table("projects")
            .select("id, organization_id")
            .order("id")
            .limit(_BATCH)
        )
        if cursor is not None:
            q = q.gt("id", cursor)
        page = q.execute()
        if not page.data:
            break
        projects.extend(page.data)
        cursor = page.data[-1]["id"]
        if len(page.data) < _BATCH:
            break

    for proj in projects:
        _seed_one(supabase, proj["organization_id"], proj["id"])

    logger.info("Seeded protocol catalog into %d projects", len(projects))


def _seed_one(supabase: Client, organization_id: str, project_id: str) -> None:
    """Copy the catalog into one project using a sync PostgREST path."""
    catalog = (
        supabase.table("experiment_templates")
        .select(
            "name, protocol_config, parameters_schema, time_series_spec,"
            " metrics_spec, plot_options, description, source_protocol"
        )
        .eq("organization_id", SYSTEM_ORGANIZATION_ID)
        .eq("access_level", "system")
        .execute()
    )
    rows = catalog.data or []
    if not rows:
        return
    payload = [
        {
            **row,
            "organization_id": organization_id,
            "project_id": project_id,
            "access_level": "project",
        }
        for row in rows
    ]
    supabase.table("experiment_templates").upsert(
        payload,
        on_conflict="content_hash,organization_id,project_id",
        ignore_duplicates=True,
    ).execute()
```

> **Consistency note:** the ops-task client is the **sync** `Client`, whereas `copy_catalog_protocols_into_project` uses the **async** `AsyncClient`. Rather than bridge event loops, this task inlines the same insert-only logic synchronously in `_seed_one` (kept deliberately identical in shape to the helper). If the `on_conflict`-on-`content_hash` decision in Task 5 Step 3 forced a per-row existence check, mirror that same fallback here.

* [ ] **Step 2: Verify prefix uniqueness + import**

Run: `cd backend && uv run python -c "import importlib; importlib.import_module('src.ops_tasks.0022_seed_project_protocols')" 2>/dev/null || echo "check module loader"` — ops tasks are numeric-prefixed; confirm the repo's ops-task runner discovers `0020_*`. Also run the pre-commit "Ops task prefixes are unique" hook implicitly on commit.
Expected: no duplicate-prefix error.

* [ ] **Step 3: Run the ops task locally and verify idempotency**

Run it via the repo's ops-task runner (check how `0018`/`0019` are invoked — likely `uv run python -m src.ops_tasks.run 0020` or a `just` recipe). Then:

```bash theme={null}
psql "$SUPABASE_DB_URL" -c "SELECT project_id, count(*) FROM public.experiment_templates WHERE access_level='project' GROUP BY project_id;"
```

Expected: each project has the catalog count. Run the task a second time → counts unchanged (idempotent).

* [ ] **Step 4: Commit**

```bash theme={null}
git add backend/src/ops_tasks/0022_seed_project_protocols.py
git commit -m "feat(protocols): ops task 0022 — backfill catalog into existing projects"
```

***

## Task 7: Seed protocols on project creation

**Files:**

* Modify: `backend/src/services/project_service.py`

* Test: `backend/tests/test_services/test_project_service.py`

* [ ] **Step 1: Write the failing tests**

Add to `test_project_service.py`. The service needs a service Supabase client to read catalog rows; check how `ProjectService` currently obtains a service client (it has `service_user_project_role_repo` — the underlying client is reachable via `.supabase`). If not cleanly reachable, add a `service_client` constructor param. Test both seed-happens and failure-surfaces:

```python theme={null}
@pytest.mark.asyncio
async def test_create_project_seeds_catalog_protocols(monkeypatch, project_service, mock_project_repos):
    # ... wire mock_project_repos so create_project + role assignment succeed
    #     (mirror the existing happy-path test in this file) ...
    seeded = {}
    async def _fake_copy(client, org, proj, created_by=None):
        seeded["args"] = (org, proj)
        return 3
    monkeypatch.setattr(
        "src.services.project_service.copy_catalog_protocols_into_project", _fake_copy
    )

    await project_service.create_project_with_admin("user-1", "org-1", _make_project_data())

    assert seeded["args"][0] == "org-1"  # org threaded through
```

```python theme={null}
@pytest.mark.asyncio
async def test_create_project_seed_failure_raises_app_error(monkeypatch, project_service, mock_project_repos):
    # ... happy-path wiring for project + role ...
    async def _boom(*a, **k):
        raise RuntimeError("catalog read failed")
    monkeypatch.setattr(
        "src.services.project_service.copy_catalog_protocols_into_project", _boom
    )
    with pytest.raises(AppError):
        await project_service.create_project_with_admin("user-1", "org-1", _make_project_data())
```

> Match the existing happy-path test's mock wiring exactly (project\_repo.create\_project, project\_role\_repo.get\_role\_by\_name returning a role with `.id`, service\_user\_project\_role\_repo.create\_user\_project\_role, project\_repo.get\_by\_id). Copy that setup rather than reinventing it.

* [ ] **Step 2: Run tests to verify they fail**

Run: `just test-backend tests/test_services/test_project_service.py -v -k "seed"`
Expected: FAIL — seeding not wired in.

* [ ] **Step 3: Wire seeding into `create_project_with_admin`**

In `project_service.py`, import the helper at top:

```python theme={null}
from src.services.protocol_catalog import copy_catalog_protocols_into_project
```

After the Project Admin role is successfully assigned and before `get_by_id` returns the project, add:

```python theme={null}
        # Seed the project with its own copies of the built-in protocol catalog.
        # Uses the service client because catalog rows live in the system org.
        try:
            await copy_catalog_protocols_into_project(
                self.service_user_project_role_repo.supabase,
                organization_id,
                project_id,
                created_by=user_id,
            )
        except Exception as e:
            logger.exception(
                "Failed to seed protocol catalog for project %s", project_id
            )
            raise AppError(
                f"Project created, but failed to seed protocols: {e}"
            ) from e
```

> Verified: `self.service_user_project_role_repo.supabase` exists (set in
> `BaseRepository.__init__`, `base.py:71`) and is the service-role `AsyncClient`
> (`get_service_user_project_role_repository` injects
> `get_supabase_async_service_client`). Use it directly — no constructor change
> needed.

* [ ] **Step 4: Run tests to verify they pass**

Run: `just test-backend tests/test_services/test_project_service.py -v`
Expected: PASS (existing tests still green).

* [ ] **Step 5: Commit**

```bash theme={null}
git add backend/src/services/project_service.py backend/tests/test_services/test_project_service.py
git commit -m "feat(protocols): seed catalog protocols on project creation"
```

***

## Task 8: Experiments upsert + finder become project-aware

**Files:**

* Modify: `backend/src/repositories/experiments.py`

* [ ] **Step 1: Update `create_experiment`**

* Add `project_id: str | None = None,` to the `create_experiment` signature.

* Add to `experiment_data`: `if project_id: experiment_data["project_id"] = project_id`.

* Change the arbiter to `on_conflict="template_id,parameters,organization_id,project_id"`.

* Pass `project_id` to the `find_by_template_and_parameters` fallback call.

* [ ] **Step 2: Update `find_by_template_and_parameters`**

* Add `project_id: str | None = None,` to the signature.

* Add `if project_id: filters["project_id"] = project_id` to the filters dict.

* [ ] **Step 3: Update callers**

Grep for `create_experiment(` and `find_by_template_and_parameters(` across the backend and thread the parent template's `project_id` where available (the experiments service and the simulate/evaluate processors). Where a caller has no project context yet during expand, leaving it None keeps the legacy org-global behaviour.

Run: `grep -rn "create_experiment(\|find_by_template_and_parameters(" backend/src`

Known caller to update: `backend/src/services/cycler_protocol_service.py` calls `find_or_create_canonical_template` (`:138`), `find_by_template_and_parameters` (`:153`), and `create_experiment` (`:164`) in `create_experiment_from_protocol` (`:77`) — thread `project_id` through all three there (both the template-side find-or-create AND the experiment-side calls), so a protocol parsed via this path lands in the right project. Confirm `create_experiment_from_protocol` has a project in scope; if not, it stays on the legacy None path for expand.

* [ ] **Step 4: Compile check**

Run: `cd backend && uv run python -c "from src.repositories.experiments import ExperimentRepository; print('ok')"`
Expected: `ok`.

* [ ] **Step 5: Run experiment-related tests**

Run: `just test-backend tests/ -v -k "experiment"`
Expected: PASS.

* [ ] **Step 6: Commit**

```bash theme={null}
git add backend/src/repositories/experiments.py
git commit -m "feat(protocols): project-aware experiments upsert arbiter + finder"
```

***

## Task 9: Route — require project\_id on create, add project\_id list filter

**Files:**

* Modify: `backend/src/routes/experiment_templates.py`

* [ ] **Step 1: Add `project_id` to the create body + require it**

In `CreateExperimentTemplateBody` add `project_id: str | None = None`. In `create_experiment_template`, at the top, raise if missing:

```python theme={null}
    if not body.project_id:
        raise BadRequestError(
            "project_id is required to create a protocol; protocols belong to a project."
        )
```

Import `BadRequestError` from `src.exceptions`. Thread `body.project_id` into both the `force_create` `template_data` (add `"project_id"` and `"access_level": "project"`) and the `find_or_create_canonical_template(...)` call (pass `project_id=body.project_id`).

* [ ] **Step 2: Add `project_id` query param to the list route**

Add `project_id: Annotated[str | None, Query()] = None` to `list_experiment_templates`. When `project_id` is provided, return that project's protocols via `get_templates_by_project_with_email(organization_id, project_id, include=include_set, ...)` and **skip the system-org union**. When absent, keep the existing org+system union (backward compatible). Preserve the `HIDDEN_TEMPLATE_NAMES` filter and the `ExperimentTemplateListResponse` shape.

* [ ] **Step 3: Also require project\_id on parse-to-template (5-layer chain)**

The parse chain is **5 layers deep**, all in
`backend/src/simulation/services/simulation_service.py` except the last:

```
route protocols.py:386
  → SimulationService.resolve_template_from_canonical   (simulation_service.py:332, static, no project_id today)
    → SimulationService._create_experiment_from_canonical (simulation_service.py:142)
      → CyclerProtocolService.create_experiment_from_protocol (cycler_protocol_service.py:77)
        → find_or_create_canonical_template (:138) / find_by_template_and_parameters (:153) / create_experiment (:164)
```

`resolve_template_from_canonical` does **not** call `find_or_create_canonical_template`
directly — two methods sit between them. Threading `project_id` requires editing
**four** signatures, not two:

1. Add a required `project_id` body field to the parse-to-template request model
   (`protocols.py`); `BadRequestError` if missing. Pass it into
   `resolve_template_from_canonical` at `:386`.
2. Add `project_id: str` to `resolve_template_from_canonical` (`:332`) and pass
   it to `_create_experiment_from_canonical`.
3. Add `project_id: str` to `_create_experiment_from_canonical` (`:142`) and pass
   it to `CyclerProtocolService.create_experiment_from_protocol`.
4. Add `project_id` to `create_experiment_from_protocol` (`:77`) and thread it
   into its three internal calls (`:138/:153/:164`).

> **Cross-reference with Task 8 Step 3**, which independently lists the same
> `cycler_protocol_service.py` calls (`:138/:153/:164`). Do steps 3–4 here and
> Task 8 Step 3 as **one coordinated edit** to `create_experiment_from_protocol`
> so `project_id` reaches both the template-side find-or-create and the
> experiment-side upsert/finder in a single pass.

* [ ] **Step 4: Run route tests**

Run: `just test-backend tests/test_routes/test_experiment_templates_list.py tests/test_routes/test_experiment_templates_conflict.py -v`
Expected: PASS. Add a test asserting missing `project_id` on create → 400 `BAD_REQUEST`, and a test asserting the project-filtered list excludes another project's row (exclusion assertion per repo rule).

* [ ] **Step 5: Commit**

```bash theme={null}
git add backend/src/routes/experiment_templates.py backend/src/routes/protocols.py backend/tests/test_routes/
git commit -m "feat(protocols): require project_id on create, project-filtered listing"
```

***

## Task 10: Frontend caller audit + full test pass

**Files:**

* Audit only (frontend); no changes expected if PR #1225 already supplies project context.

* [ ] **Step 1: Audit create/parse callers for project\_id**

Run: `grep -rn "experiment_templates\|parse-to-template\|experimentTemplate" frontend/src --include="*.ts" --include="*.tsx" | grep -iE "post|create|parse"`
Confirm each create/parse call site sends a `project_id`. If any straggler doesn't (and is a legitimate projectless caller), it stays on the legacy org path — but a create call without project\_id will now 400, so fix those to pass the in-project `project_id`. Report findings; make minimal frontend edits only where a create/parse call is missing project\_id.

* [ ] **Step 2: Full backend test suite**

Run: `just test-backend -v`
Expected: green. Investigate any failure touching experiments/templates/projects.

* [ ] **Step 3: Frontend unit + type check (if any frontend edits were made)**

Run: `just test-frontend-unit` and `cd frontend && npx tsc --noEmit`
Expected: green.

* [ ] **Step 4: Commit any frontend fixes**

```bash theme={null}
git add frontend/src
git commit -m "fix(protocols): pass project_id on protocol create/parse from in-project UI"
```

***

## Task 11: Browser verification of the in-project protocol flow

* [ ] **Step 1: Start servers + log in** per `CLAUDE.local.md` (preview\_start frontend/backend, inject auth token).
* [ ] **Step 2: Open a project's protocol list**, confirm the seeded catalog protocols appear.
* [ ] **Step 3: Create a protocol in the project**, confirm it saves and appears in that project only (open a second project, confirm it is NOT there — exclusion check).
* [ ] **Step 4: Screenshot** the seeded list + the created protocol as proof.

> Worktree gotchas (from memory): vite `--port N --strictPort`; backend `.env` SUPABASE\_URL must point at the running stack (54321/54322); if jobs run, `RAY_DASHBOARD_PORT=8284`.

***

## Final: open the PR

* [ ] Run the `github-pr-labels` + `git-workflow` conventions; open PR titled `feat(protocols): scope protocols to project_id (expand)` describing the expand phase and linking the spec. Note the two follow-ups (contract PR ≥1mo later; RLS PR after that). Reference that ops task `0022` must be run per-environment.